Source-linked AI summary
An Ethical Trajectory Planning Algorithm for Autonomous Vehicles
Maximilian Geisslinger, Franziska Poszler, Markus Lienkamp
TL;DR
Ethical AV trajectory planning must address how risk and harm are distributed among road users. The paper proposes a framework and trajectory-planning algorithm combining ethical principles; simulations report risk shifting from vulnerable road users to the ego vehicle compared with selfish approaches.
Problem
Estimating harm for ethical trajectory planning raises ethical questions, while the framework seeks fair risk distribution among road users.
Source: Risk Calculation · p. 3, Discussion · p. 9Method
The authors propose an ethical trajectory-planning framework that combines risk modeling with principles including responsibility and maximum acceptable risk.
Source: Responsibility Principle · p. 6, Discussion · p. 9Results
Simulations with an exemplary equal-weight parameter set show risk shifting from vulnerable road users to the autonomous ego vehicle compared with selfish approaches.
Source: Discussion · p. 9Takeaways & Limitations
The proposed principles apply beyond potentially hazardous situations to varied driving situations and support transparent decision-making.
Source: Discussion · p. 9The framework does not specify principle weights or a maximum acceptable risk value, and the authors say these parameters require social consensus and further research.
Source: Discussion · p. 9
Abstract
from arXiv · showhide
With the rise of AI and automation, moral decisions are being put into the hands of algorithms that were formerly the preserve of humans. In autonomous driving, a variety of such decisions with ethical implications are made by algorithms for behavior and trajectory planning. Therefore, we present an ethical trajectory planning algorithm with a framework that aims at a fair distribution of risk among road users. Our implementation incorporates a combination of five essential ethical principles: minimization of the overall risk, priority for the worst-off, equal treatment of people, responsibility, and maximum acceptable risk. To the best of the authors' knowledge, this is the first ethical algorithm for trajectory planning of autonomous vehicles in line with the 20 recommendations from the EU Commission expert group and with general applicability to various traffic situations. We showcase the ethical behavior of our algorithm in selected scenarios and provide an empirical analysis of the ethical principles in 2000 scenarios. The code used in this research is available as open-source software.
The Algorithm
The algorithm samples trajectories and assigns road-user-specific risks, then ranks trajectories through validity levels and ethical costs. It executes the lowest-cost trajectory in the highest available validity level and replans after each time step.
The Algorithm
- It samples multiple trajectories from the AV's current state and assigns a risk value for every road user on each trajectory.This work generates trajectories in a Frenet coordinate system.Source: The Algorithm · p. 2
- Sampled trajectories are grouped into four validity levels, preserving a solution when not all quality requirements can be met.Source: The Algorithm · p. 2
- For trajectories in the highest available validity level, an ethical cost function promotes fair risk distribution.The cost function depends on the validity level.Source: The Algorithm · p. 2
- The algorithm executes the lowest-cost trajectory from the highest available validity level, then replans after one time step.Source: The Algorithm · p. 2
Risk Calculation
Risk is treated as expected harm: collision probability multiplied by estimated personal injury. The method predicts collision probabilities, estimates harm, and assigns trajectory-level risks to each road user, while making specified modeling and aggregation choices.
Risk Calculation
- Risk is defined as collision probability multiplied by estimated harm, with harm representing expected personal injury rather than property damage.The harm scale runs from 0 for a collision without human harm to 1 for the greatest possible harm; the model excludes personal characteristics such as age, gender, or quality-of-life measures.Source: Risk Calculation · p. 3
- A neural network predicts other road users’ trajectories as bivariate normal distributions around their most-likely paths, supporting collision-probability estimates for planned AV states.The probability calculation uses the object shapes.Source: Risk Calculation · p. 3
- The harm model distinguishes protected from unprotected road users and uses CRSS-based logistic regressions, with MAIS3+ accident severity as a comparison value.The model uses mass, velocity, collision angle, and impact area; impact areas are aligned symmetrically to mitigate co-driver bias.Source: Risk Calculation · p. 3
- The model represents a trajectory’s risk to each road user by its maximum risk over time, accounting for non-independent risks along the planning horizon.This yields a risk value for every road user conditional on the AV trajectory.Source: Risk Calculation · pp. 3–4, Figure 2 · p. 4
- Risks from different collisions are assumed independent when combined; for the maximum acceptable risk principle, combined multi-user collision risk is calculated only for the ego-vehicle.The latter choice is attributed to information asymmetry.Source: Risk Calculation · p. 4
Maximum Acceptable Risk
The framework sets a maximum acceptable risk threshold that determines which trajectories are allowed and which are reserved for cases with no alternative. This supports maneuver-level decisions and makes the risk threshold a transparent question for public debate.
Maximum Acceptable Risk
- A trajectory that exceeds the maximum acceptable risk is permitted only when no other option is available, so the selected path must come from the highest available validity level.In the cyclist scenario, overtaking is disallowed because staying behind is a valid alternative.Source: Maximum Acceptable Risk · p. 4, Figure 3 · p. 4
- The threshold makes maneuver-level choices possible, including whether to perform a potentially dangerous overtaking maneuver.Source: Maximum Acceptable Risk · p. 4
- The authors argue that determining the threshold concerns all road users, not only AV users.Source: Maximum Acceptable Risk · p. 4
- A risk threshold can improve decisions if AV risk assessment is reliable and make AV functioning more transparent to traffic participants.The framework presents the threshold as a basis for social debate about acceptable risk in exchange for mobility.Source: Maximum Acceptable Risk · p. 4
- Human-driver safety offers an initial indication for a road-traffic threshold, but widespread acceptance may require a significantly higher safety level.Source: Maximum Acceptable Risk · p. 5
Risk Distribution
For valid trajectories, the cost function balances risk, mobility, and comfort; when no valid trajectory exists, safety takes priority. The risk cost combines four ethical principles to guide risk distribution.
Risk Distribution
- For valid trajectories, total trajectory cost combines risk, mobility, and comfort, while the cost function changes with the highest available validity level.Source: Risk Distribution · p. 5
- When no valid trajectory can be generated, risk alone is considered until valid trajectories become available again.This fallback gives risk higher priority than mobility and comfort because of the possible accident threat.Source: Risk Distribution · p. 5
- The risk cost function combines Bayes, equality, maximin, and responsibility principles in a weighted sum to represent fair risk distribution.The responsibility term reduces cost consideration for responsible road users, but its weight prevents increasing their risks without benefit.Source: Risk Distribution · p. 5
Bayes, Equality & Maximin Principle
The Bayes principle minimizes average overall risk, while equality and maximin add fairness by addressing unequal risk and the greatest possible harm.
Bayes Principle
- The Bayes principle selects the trajectory with the lowest normalized total risk across detected road users.Normalizing by the number of road users keeps the cost independent of how many are present.Source: Bayes, Equality & Maximin Principle · p. 5
Equality Principle
- The equality principle penalizes differences in risk among road users, complementing Bayes minimization with equal treatment.Its normalization keeps the cost independent of the number of road users.Source: Bayes, Equality & Maximin Principle · pp. 5–6
Maximin Principle
- Maximin prioritizes the worst-off by minimizing the greatest possible harm, separately from collision probability.A discount factor γ≥1 addresses the principle’s neglect of usually low probabilities.Source: Bayes, Equality & Maximin Principle · p. 6
Responsibility Principle
The responsibility principle reduces the risk cost assigned to road users who contribute to risky situations, using rule adherence, road-user characteristics, and available actions to assess responsibility.
Motivation
- Considering responsibility is presented as a way to address fairness and encourage responsible behavior by human road users.The paper connects this aim to concerns about pedestrians exploiting crash avoidance and to the EU Commission’s call for a culture of responsibility.Source: Responsibility Principle · p. 6
Responsibility Principle
- Responsibility reduces the Bayes risk cost assigned to responsible road users, but the responsibility factor is kept below 1.A value of 1 would remove that road user’s risk cost from the Bayes term entirely.Source: Responsibility Principle · p. 6
- The principle considers three responsibility signals: traffic-rule adherence, road-user characteristics, and the space of actions.The rule-violation component’s responsibility amount remains an open question for future research.Source: Responsibility Principle · pp. 6–7
- Rule-based responsibility uses each road user’s reachable set of physically and legally possible behaviors to assess whether a collision involves a violation.A collision outside a road user’s reachable set can indicate responsibility; the amount depends on violation type and severity, which requires further research.Source: Responsibility Principle · pp. 6–7, Figure 4 · p. 7
- The algorithm also assigns motorized road users some responsibility based on mass and the risk they bring to traffic, regardless of behavior.This reflects their special responsibility to protect vulnerable road users.Source: Responsibility Principle · p. 7
- Responsibility can also reflect each vehicle’s available actions, such as assigning responsibility to the vehicle behind in a rear-end collision.The algorithm incorporates this kind of responsibility using reachable sets.Source: Responsibility Principle · p. 7
Results
Across 2,000 CommonRoad simulations, ethical planning reduced risk for third parties—especially vulnerable road users—while keeping ego-vehicle risk similar to selfish planning. Simulated accident harm broadly confirmed the risk-distribution findings.
Evaluation
- The evaluation compared ethical, selfish, and standard algorithms in 2,000 CommonRoad scenarios spanning varied environments and regions.Traffic participants behaved deterministically and did not interact with the AV, so the evaluation excluded the responsibility principle.Source: Results · p. 8
- The analysis focused on the 100 highest risk values for the ego AV, third parties, and vulnerable road users because most other values were close to zero.It assessed both risk distributions and simulated accidents.Source: Results · p. 8, Figure 5 · p. 8
Results
- Ethical planning reduced risk for all road-user groups versus the standard algorithm and lowered third-party, especially VRU, risk versus selfish planning at similar ego risk.The VRUs nevertheless face higher risk because they experience greater harm if an accident occurs.Source: Figure 5 · p. 8, Results · p. 8
- In 2,000 simulations, observed accident harm broadly confirmed the risk-distribution findings, though ethical and selfish harm was similar for third-party road users.VRUs were less prominent within third-party harm than in the risk distribution; discrepancies were attributed to risk-model inaccuracies and few accidents.Source: Figure 6 · p. 8, Results · p. 8
Discussion
Equal-weight simulations show the algorithm shifting risk from vulnerable road users to the autonomous vehicle, while the authors identify unresolved parameter choices and limits in risk modeling. They argue ethical decision-making should be incorporated despite those limits and note that human drivers do not use the same kind of risk-based reasoning.
Discussion
- The authors present risk-based principles as applicable beyond hazardous situations, with transparent decisions and responsibility that may account for long-term effects.They suggest the principles may also interest future AI applications beyond autonomous driving.Source: Discussion · p. 9
- The framework leaves principle weights and the maximum acceptable risk unspecified because parameter choices may not apply universally and should follow social consensus.The authors propose user studies to examine parameter preferences and help validate the risk-distribution principles.Source: Discussion · p. 9
- In equal-weight simulations, the fair-risk algorithm shifts risk from vulnerable road users to the autonomous ego vehicle, unlike selfish approaches.The authors note that the model uses only uncertainties from predicting other road users.Source: Discussion · p. 9
- Risk modeling is limited by available information, and the authors say perfect all-encompassing ethical solutions are unlikely and errors may occur.They illustrate the complexity of responsibility with different reasons a pedestrian might run into the street.Source: Discussion · p. 9
- Human drivers primarily act instinctively rather than performing the algorithm's kind of risk-based reasoning, which the authors say is not a reason to reject ethical AV software.Source: Discussion · p. 9
Data availability
The research data, including evaluation files for simulated scenarios using various algorithms, are available in the Supplementary data file.
Data availability
- The Supplementary data file contains the research data, including evaluation files for simulated scenarios using various algorithms.Source: Data availability · p. 9
Fundig
The authors acknowledge financial support from the Technical University of Munich's Institute for Ethics in Artificial Intelligence and state that the views expressed are their own.
Fundig
- The Technical University of Munich—Institute for Ethics in Artificial Intelligence provided financial support; the opinions and conclusions do not necessarily reflect its or its partners' views.Source: Fundig · p. 9