Source-linked AI summary
Machine Ethics and Automated Vehicles
Noah J. Goodall
TL;DR
Automated vehicles face ethically complex risk-allocation decisions, especially when crashes cannot be avoided and no human oversees them. This chapter motivates research through anticipated critiques and related work, concluding that vehicles must sometimes make ethical choices autonomously and much work remains.
Problem
Method
The chapter argues for automated-vehicle ethics by responding to anticipated critiques and discussing applications from machine ethics and moral modeling.
Source: §1 · p. 1, §2 · p. 3, §3.2 · p. 8Results
The chapter concludes that automated vehicles must sometimes make ethical choices autonomously, while machine ethics and moral modeling have made progress but much work remains.
Source: §4 · p. 9Takeaways & Limitations
Automated-vehicle ethics will likely require a combination of two or more ethical theories.
Source: §3.1 · p. 7Preliminary evidence does not prove automation is safer than human drivers, and an overall fatality reduction could still coincide with increased cyclist fatalities.
Source: Criticism 4: Automated vehicles will… · p. 6
Abstract
from arXiv · showhide
Road vehicle travel at a reasonable speed involves some risk, even when using computer-controlled driving with failure-free hardware and perfect sensing. A fully-automated vehicle must continuously decide how to allocate this risk without a human driver's oversight. These are ethical decisions, particularly in instances where an automated vehicle cannot avoid crashing. In this chapter, I introduce the concept of moral behavior for an automated vehicle, argue the need for research in this area through responses to anticipated critiques, and discuss relevant applications from machine ethics and moral modeling research.
1 Ethical Decision Making for Automated Vehicles
This chapter focuses on machine ethics for automated vehicles, which must make ethically complex driving decisions and autonomously accept and apportion risk because travel cannot be completely safe.
1 Ethical Decision Making for Automated Vehicles
- The chapter addresses ethically complex automated-vehicle decisions, particularly before a crash, rather than the separate unresolved question of crash liability.Source: §1 · p. 1
- Road travel cannot be completely safe: a tractor trailer needs eight seconds and a passenger car three seconds to stop, while accurate prediction of nearby vehicles over that interval is impossible.Source: §1 · p. 1
- Automated vehicles must continually assess driving risks, decide what risk to accept for themselves and adjacent vehicles, and determine how to apportion it.Crash-time constraints mean these decisions must be made autonomously.Source: §1 · pp. 1–2
2 Criticisms of the Need for Automated Vehicle Ethics Systems, and Responses
The section argues that automated vehicles will face morally or legally ambiguous situations and that ethical decisions remain necessary wherever driving involves risk. It answers nine criticisms by identifying crash risks, limits of human control and law, and ethical concerns that aggregate safety gains alone cannot resolve.
2 Criticisms of the Need for Automated Vehicle Ethics Systems, and Responses
- Crashes remain possible with imperfect systems and even perfect systems, because road traffic includes unpredictable hazards and unavoidable collision states.Imperfect systems can fail through hardware, software, perception, or reasoning errors. In mixed traffic, avoiding every possible collision state is impossible, while wildlife, pedestrians, and bicyclists remain threats even among automated vehicles.Source: §2 · pp. 2–3
- Ethical choices are not limited to rare trolley-style dilemmas: everyday risk decisions become complex under uncertainty about outcomes, probabilities, and the vehicle’s surroundings.Google’s vehicle team reportedly considered moral questions in avoiding debris, including whether to avoid a cat, deer, or child.Source: §2 · pp. 3–4
- An automated vehicle’s ethical responsibility to protect occupants and others does not depend on whether it is legally at fault for a crash.Source: §2 · p. 4
Criticism 4: Automated vehicles will never collide with another automated
- Automated vehicles will continue to encounter human road users and other hazards, since fleet turnover is slow and even automated-only zones cannot eliminate risks.Even if every new U.S. vehicle sold were automated, 90% replacement would take 30 years.Source: Criticism 4: Automated vehicles will… · p. 4
- Drivers in level 2 and 3 vehicles may not be ready to take over in time, leaving the automated vehicle responsible for emergency ethical decisions.AASHTO recommends 200 to 400 meters for perception and reaction at 100 km/hr, corresponding to 7 to 14 seconds; the required warning time remains unspecified.Source: Criticism 4: Automated vehicles will… · pp. 4–5
- Driving routinely involves ethical decisions that transfer risk, including speeding or giving a cyclist room, and novel situations should arise steadily across 4.8 trillion kilometers of annual U.S. driving.Source: Criticism 4: Automated vehicles will… · p. 5
- Following the law alone cannot ensure reasonable behavior: current laws lack sufficient specificity, and encoding enough computer-understandable rules would be a massive undertaking.For example, a vehicle could be forced to wait at a tree branch rather than cross a double yellow line when no traffic is oncoming.Source: Criticism 4: Automated vehicles will… · p. 5
- Minimizing damage at all times may maximize safety but can violate individual rights and be considered unfair when it selects whom to endanger.Expected aggregate safety gains do not settle ethics: automation’s safety benefits remain unproven, may redistribute harm between groups, and must fit social values to earn public trust. A vehicle would need to travel 1.1 million kilometers without crashing and 482 million kilometers without a fatal crash to be deemed safer than a human driver with 99% confidence.Source: Criticism 4: Automated vehicles will… · pp. 5–6
3 Relevant Work in Machine Ethics and Moral Modeling
Machine ethics addresses two challenges for automated vehicles: articulating society’s values across uncertain, layered scenarios and translating those values into computer-readable form. This section reviews ethical theories and computational tools that model moral decisions.
3.1 Ethical Theories
- Deontological rules risk literal misinterpretation and cannot plausibly cover every situation, while utilitarianism is computable but difficult to measure and can sacrifice equity; vehicle ethics will likely need multiple theories.Utilitarianism may favor colliding with a helmeted motorcyclist over a non-helmeted one when damage costs define utility.Source: §3.1 · p. 7
3.2 Practical Applications
- Jeremy estimates action utility from intensity, duration, and probability, but its exclusively utilitarian approach omits context, virtues, and limits on individual harm.Source: §3.2 · pp. 7–8
- W.D. and MedEthEx use machine learning on duty-based test cases, calibrated by expert judgment, to classify actions as right or wrong and identify key principles.Source: §3.2 · p. 8
- Truth-Teller and SIROCCO use casuistic comparisons to surface relevant reasons or ethical principles for users without making moral judgments.Truth-Teller compares truth-telling cases; SIROCCO identifies relevant principles from the engineering ethics code.Source: §3.2 · p. 8
- For battlefield decisions, the Metric of Evil weights consequences using estimates, confidence intervals, intentionality, and expert-rated cases, but provides no evidence for its conclusions.Source: §3.2 · p. 8
- Computational moral modeling remains in its infancy: existing tools demonstrate automatic ethical assessment, but model calibration and uncertainty remain major challenges.Source: §3.2 · p. 8
4 Summary
Automated vehicles will continue to crash, requiring them to assess and apportion risk to themselves and others without human oversight. This chapter frames the resulting ethical problem and points to research directions in machine ethics and moral modeling.
4 Summary
- Automated vehicles will continue to crash and must continually assess whether risks to themselves and others are acceptable.Even simple maneuvers require risk assessment.Source: §4 · p. 9
- Risk acceptance and apportionment are ethical decisions that vehicles must sometimes make autonomously, using pre-programmed instructions, machine learning, or both.Human drivers will not be able to oversee these decisions.Source: §4 · p. 9
- Moral modeling and machine ethics have made some progress, but substantial work remains to guide research on ethical systems for automated vehicles.The chapter introduces core concepts and directions for useful research in related fields.Source: §4 · p. 9