Source-linked AI summary
SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies
Matt Vitelli, Yan Chang, Yawei Ye, Maciej Wołczyk, Błażej Osiński, Moritz Niendorf, Hugo Grimmett, Qiangui Huang, Ashesh Jain, Peter Ondruska
TL;DR
Self-driving vehicles still struggle to safely handle rare road events, while traditional planners require costly scenario-specific engineering and ML planners lack safety guarantees. SafetyNet combines an ML planner with a rule-based fallback that checks and can minimally modify its decisions. The paper reports improved safety and comfort metrics over an ML-only system in simulation and on challenging San Francisco streets.
Problem
Self-driving vehicles can behave unsafely in rarely occurring road scenarios, while traditional planning requires costly scenario-specific engineering and ML planning does not provide safety guarantees.
Source: §I · p. 1Method
SafetyNet uses an ML planner trained from expert demonstrations and a lightweight rule-based fallback that checks its intended trajectory and selects a safe candidate when required constraints are violated.
Source: §I · p. 1, §III · p. 3Results
SafetyNet showed significant improvements in safety and comfort metrics over a purely ML-based system in simulation and on challenging San Francisco streets.
Source: §V · p. 6Takeaways & Limitations
The authors conclude that a fallback layer makes it possible to use learned planners in the real world while retaining their ability to improve with data and handle complex situations.
Source: §V · p. 6
Abstract
from arXiv · showhide
In this paper we present the first safe system for full control of self-driving vehicles trained from human demonstrations and deployed in challenging, real-world, urban environments. Current industry-standard solutions use rule-based systems for planning. Although they perform reasonably well in common scenarios, the engineering complexity renders this approach incompatible with human-level performance. On the other hand, the performance of machine-learned (ML) planning solutions can be improved by simply adding more exemplar data. However, ML methods cannot offer safety guarantees and sometimes behave unpredictably. To combat this, our approach uses a simple yet effective rule-based fallback layer that performs sanity checks on an ML planner's decisions (e.g. avoiding collision, assuring physical feasibility). This allows us to leverage ML to handle complex situations while still assuring the safety, reducing ML planner-only collisions by 95%. We train our ML planner on 300 hours of expert driving demonstrations using imitation learning and deploy it along with the fallback layer in downtown San Francisco, where it takes complete control of a real vehicle and navigates a wide variety of challenging urban driving scenarios.
I. INTRODUCTION
SafetyNet addresses the challenge of safely handling rare driving events by combining a data-scalable ML planner with a rule-based fallback. The introduction reports that the combination outperforms ML-only systems and was deployed on busy San Francisco streets.
I. INTRODUCTION
- Rare road events challenge SDVs, while traditional planning requires scenario-specific loss engineering and recent demonstration-trained ML policies lack the safety guarantees needed for deployment..Source: §I · p. 1
- SafetyNet checks ML-planner decisions against a small set of rules and can minimally modify them to enforce safety and legality constraints while maximizing comfort metrics.The checks include collision avoidance and road-rule violations.Source: §I · p. 1
- The combination outperforms ML-only systems and was deployed on busy San Francisco streets, demonstrating varied maneuvers without impeding the safety of the vehicle and other traffic participants.Reported maneuvers include lane-following, keeping distance from other vehicles, and navigating intersections.Source: §I · pp. 1–2
- The authors identify the first combination of machine learning and a lightweight hand-engineered system for SDV control with safety and legality guarantees, and the first evaluation of such a system in downtown San Francisco.They also state that source code will be made public.Source: §I · p. 2
II. RELATED WORKS
Related work contrasts hand-engineered trajectory optimization with data-scalable ML planning, which avoids costly rule design but lacks safety guarantees. Existing hybrid and safety-framework approaches motivate a simpler fallback aimed at deployment rather than a comprehensive safety framework.
Trajectory optimization-based planning
- Trajectory-optimization planners minimize hand-engineered costs, but designing a human-like comfort, safety, and progress trade-off is difficult and engineering effort grows across scenarios and geographies.Such systems also generalize poorly to highly unstructured urban scenarios.Source: §II · p. 2
Machine-learned planning
- ML planning scales with data, but its lack of safety guarantees has limited real-world deployment; prior work had not yet been evaluated in real-world urban driving.Earlier imitation-learning demonstrations were limited to simple tasks or had not been tested in real-world urban environments; cited RL and IRL methods were also unevaluated there.Source: §II · p. 2
Hybrid approaches
- Hybrid planning includes ML-based heuristics and modular systems that score expert-generated trajectories, but the cited collision-cost guarantees had not been verified in the real world..Source: §II · p. 2
Safety frameworks
- SafetyNet is distinguished from comprehensive safety-framework work: its stated aim is a simple method for deploying a powerful neural planner while enforcing certain safety and legality constraints.The system relies on an expert system for selected deterministic, legal, and safe scenarios and on ML for nominal trajectory generation.Source: §II · pp. 2–3
III. HYBRID ML PLANNING SYSTEM
SafetyNet combines an expert-demonstration-trained ML planner with a fallback layer that checks interpretable safety constraints. The fallback validates the intended trajectory and selects the closest safe candidate if the constraints are not met.
III. HYBRID ML PLANNING SYSTEM
- The system learns trajectory planning from expert driving demonstrations while enforcing interpretable constraints such as collision avoidance and adherence to traffic rules..Source: §III · p. 3
- The ML policy maps the environment state to an intended trajectory, which is validated; if it violates required constraints, the system selects the closest trajectory from safe candidates..Source: §III · p. 3
A. Input and Output
The planner represents its surroundings in an ego-centric frame and predicts a trajectory as a sequence of timed vehicle states. Open-loop prediction error improves with more training data, while the fallback layer improves the reported safety and comfort metrics except passiveness.
A. Input and Output
- The input uses an ego-centric frame and includes the SDV, perceived agents, static and dynamic map elements, and the intended route.The listed elements include their pose relative to the SDV and additional features such as element type and observation time.Source: A. Input and Output · p. 3
- A trajectory is a sequence of T states spaced uniformly by Δt, with each state encoding rear-axle pose, velocity, acceleration, curvature, and jerk.The pose is measured relative to a fixed coordinate frame.Source: A. Input and Output · p. 3
- The fallback layer significantly improves every plotted safety and comfort metric except passiveness, while collisions decline slowly as the planner gets more data.The figure reports safety events in the top plots and comfort events in the bottom plots.Source: Figure 3 · p. 3
- Open-loop prediction error improves when the planner is trained with more data.Table I presents the open-loop performance trend as training-set size increases.Source: A. Input and Output · p. 3, Table I · p. 3
B. ML planner
The ML planner maps scene inputs to an executable trajectory using a hierarchical graph network and a kinematic decoder. It is trained by imitation learning, with perturbations and penalties intended to address covariate shift and improve comfort.
B. ML planner
- The ML planning component takes scene-state input I and outputs a trajectory for execution.Source: B. ML planner · p. 3
- A PointNet-based local subgraph processes vectorized inputs, while a Transformer-based global graph reasons over agents and map features.The architecture is hierarchical and graph-network-based.Source: B. ML planner · p. 3
- A kinematic decoder predicts longitudinal jerk and curvature, then rolls out the ego vehicle’s next state from those predictions and its current state.Vehicle-kinematic constraints clip controls to enforce physical feasibility.Source: B. ML planner · pp. 3–4
- The training framework uses imitation learning to minimize L1 pose error and perturbs training states to reduce the impact of covariate shift.Penalties on jerk and curvature are used to reduce jerk and improve driving comfort.Source: B. ML planner · p. 4
C. Fallback Layer
The fallback layer labels ML trajectories feasible or infeasible using dynamic, traffic-rule, and collision checks. It executes feasible ML trajectories and otherwise selects the feasible generated candidate most similar to the ML trajectory.
C. Fallback Layer
- The system evaluates each ML trajectory for dynamic feasibility, legality, and collision likelihood before assigning a feasible or infeasible label.Source: C. Fallback Layer · p. 4
- Dynamic checks test jerk, acceleration, curvature, curvature rate, lateral acceleration, and steering jerk against bounds derived from real-world vehicle testing.Jerk, longitudinal acceleration, and lateral-acceleration limits are typically made more conservative for comfort.Source: C. Fallback Layer · p. 4
- A trajectory is infeasible if it runs a stop sign or red light, violates right of way, or leaves the drivable surface.Source: C. Fallback Layer · p. 4
- Collision checks compare planned ego poses with predicted poses of other agents and also test longitudinal distance, time to collision, and time headway.Failure of any collision-likelihood check labels the trajectory infeasible.Source: C. Fallback Layer · p. 4
- The system executes a feasible ML trajectory directly; if it is infeasible, it selects the feasible fallback trajectory closest to it.Generated lane-aligned candidates include speed keeping, distance keeping, and emergency stopping maneuvers.Source: C. Fallback Layer · p. 4
IV. EXPERIMENTS
The evaluation covers the ML planner’s simulation performance as training data grows, the fallback layer’s effectiveness in simulation, and SafetyNet’s real-world performance controlling a vehicle in San Francisco.
IV. EXPERIMENTS
- The evaluation measures the ML planner in simulation as it is trained on increasing amounts of data, without the fallback layer.Source: §IV · p. 4
- A separate simulation evaluation examines the effectiveness of the fallback layer.Source: §IV · p. 4
- The evaluation also tests SafetyNet controlling a real vehicle in San Francisco.Source: §IV · p. 4
A. Data
The evaluation uses an urban-driving dataset and closed-loop simulation to assess SafetyNet’s data, safety, comfort, and trajectory-similarity measures, then tests deployment in downtown San Francisco.
A. Data
- The in-house dataset contains 380 hours of urban driving from Palo Alto and San Francisco, partitioned into 300 hours for training and 80 for testing.Scenes are 25 seconds long and include perception output, the SDV trajectory, and HD maps.Source: A. Data · p. 4
B. Metrics
- Closed-loop simulation unrolls the policy through each test scene using a vehicle model calibrated to the real SDV and reactive longitudinal behavior for other agents.Other agents preserve their logged trajectories while avoiding collisions.Source: B. Metrics · p. 4
- Table II compares ML-Planner-only and SafetyNet performance in closed-loop evaluation, but the supplied caption gives no cell values.The evaluation also reports that SafetyNet reduces collisions, close calls, and discomfort braking at the expense of more passiveness.Source: Table II · p. 5, B. Metrics · p. 5
- Safety and comfort events include collisions, close calls, discomfort braking, passiveness, and off-road events, aggregated per 1,000 simulated miles.Close calls include proximity within 25 cm, time-to-collision below 1.5 seconds, or time headway below 1 second; discomfort braking is jerk below −5 m/s^3.Source: B. Metrics · pp. 4–5
- Average Displacement Error (ADE) measures open-loop trajectory similarity between simulated and dataset positions.The evaluation also tests the deployed system in downtown San Francisco, using fallback-trajectory frequency and qualitative examples to assess planner performance.Source: B. Metrics · p. 5
C. Effect of dataset size
In simulation without fallback, closed-loop performance improves as the ML planner receives more training data, though passiveness plateaus at 300 hours.
C. Effect of dataset size
- Closed-loop safety and comfort metrics improve as training data increases, with performance improving significantly at 300 hours.The compared dataset sizes are 5, 50, 150, and 300 hours; models trained on under 50 hours show more collisions, off-route events, or passiveness.Source: C. Effect of dataset size · p. 5
- At 300 hours, the learned policy reduces collisions, close calls, and discomfort braking while reliably following the route.Passiveness plateaus, which the authors attribute to causal confusion inherent to open-loop training.Source: C. Effect of dataset size · p. 5
- Open-loop Average Displacement Error continues to improve with more training data.The authors also note that further closed-loop improvements with more data may be slow.Source: C. Effect of dataset size · p. 5
D. Effect of fallback layer
In simulation, the fallback layer substantially reduces safety and comfort events while making driving more passive; it intervenes when the ML planner’s trajectory fails checks.
D. Effect of fallback layer
- Collisions fall by 95%, close-call events by 40%, and discomfort braking by 92% when the fallback layer is enabled.The comparison uses the ML planner trained on the full 300-hour dataset and evaluates closed-loop performance.Source: D. Effect of fallback layer · p. 5
- Passiveness rises by 29.5% with fallback enabled, a trade-off the authors attribute to more conservative driving and regard as necessary for reducing collisions and close calls.Source: D. Effect of fallback layer · pp. 5–6
- Fallbacks address trajectories that leave the drivable surface, contact predicted agents or static obstacles, create infeasible gaps, or exceed feasible steering jerk.Examples include preventing a bus collision, a red-light violation, and failure to yield to oncoming traffic; the latter is attributed to synthetic ego-pose perturbations during training.Source: Figure 5 · p. 5, D. Effect of fallback layer · p. 6
E. Failure cases
SafetyNet still misses a small proportion of planner failures, but it was tested on public roads in San Francisco and the authors conclude that the combined system enables safe use of learned planners in real-world urban settings.
E. Failure cases
- A small proportion of ML planner failures escape detection when the vehicle is near a lane boundary and an oncoming vehicle suddenly changes speed.The fallback does not limit proximity to lane boundaries or fully compensate for uncertainty in other agents’ behavior predictions, so lateral clearance may be uncomfortable.Source: E. Failure cases · p. 6
F. Real world testing
- During 150+ miles of supervised public-road testing in downtown San Francisco, the 300-hour planner handled challenging maneuvers, while fallback control was used for around 7.9% of driving time.The planner drove safely for most of the time (>90%), according to the authors.Source: F. Real world testing · p. 6, Figure 7 · p. 6
V. CONCLUSIONS
- The authors conclude that SafetyNet combines learned planning with a rule-based fallback to support safe driving in challenging real-world urban environments.They report improved safety and comfort metrics over a purely ML-based system in simulation and San Francisco streets, while noting that the fallback could be refined to reduce passiveness and the imitation-learning planner could be improved.Source: §V · p. 6