Source-linked AI summary
Machine Ethics and Automated Vehicles
Noah J. Goodall
TL;DR
Automated vehicles must manage unavoidable driving risk without dependable human oversight, making crash avoidance and risk allocation ethical problems. This chapter motivates research on moral behavior by addressing objections, reviewing ethical theories and moral-modeling applications, and concludes that substantial work remains.
Problem
Automated vehicles must make ethical risk-allocation decisions when crashes cannot be avoided and human drivers cannot oversee them.
Method
The chapter addresses anticipated criticisms, reviews ethical theories and moral-modeling research, and discusses practical applications to automated vehicles.
Results
The chapter concludes that automated vehicles will continue to crash and that existing moral-modeling and machine-ethics research has made progress but requires substantial further work.
Takeaways & Limitations
Automated vehicles may require explicit rules, machine learning, or a combination to make ethical choices autonomously.
Takeaways & Limitations
Deontological rules can produce unexpected behavior because computers may interpret complex human values and common-sense rules literally.
Abstract
from arXiv · showhide
Road vehicle travel at a reasonable speed involves some risk, even when using computer-controlled driving with failure-free hardware and perfect sensing. A fully-automated vehicle must continuously decide how to allocate this risk without a human driver's oversight. These are ethical decisions, particularly in instances where an automated vehicle cannot avoid crashing. In this chapter, I introduce the concept of moral behavior for an automated vehicle, argue the need for research in this area through responses to anticipated critiques, and discuss relevant applications from machine ethics and moral modeling research.
1 Ethical Decision Making for Automated Vehicles
Automated vehicles face unavoidable risk even with advanced sensing and control, requiring autonomous ethical decisions about accepting and allocating that risk. The chapter frames these decisions as a central problem for vehicle automation research.
- 1 Ethical Decision Making for Automated Vehicles: Automated driving cannot eliminate all risk because vehicles need accurate predictions of nearby road users over stopping distances that are not always possible.At 100 km/hr, a loaded tractor trailer requires eight seconds to stop and a passenger car requires three seconds.
- 1 Ethical Decision Making for Automated Vehicles: Vehicles must continually assess risks associated with speed, road position, passing, and collision avoidance.Examples include taking a curve at a chosen speed, crossing the centerline to pass a cyclist, or moving toward another vehicle to avoid a truck.
- 1 Ethical Decision Making for Automated Vehicles: Automated vehicles must decide how much risk to accept for themselves and adjacent vehicles, then apportion that risk among affected parties.Because crash decisions occur under time constraints, these ethical choices must be made autonomously.
- 1 Ethical Decision Making for Automated Vehicles: The chapter responds to nine criticisms of ethics research, reviews ethical theories and moral modeling, and concludes with a summary.Its organization moves from the need for ethical decision systems to relevant theoretical and practical research.
2 Criticisms of the Need for Automated Vehicle Ethics Systems, and Responses
The chapter argues that ethics research remains necessary because automated vehicles may still crash, face ethically complex uncertainty, and operate without reliable human intervention. Existing law, safety improvements, and utilitarian calculations do not by themselves resolve these decisions fairly or comprehensively.
- Criticism 1: Automated vehicles will never (or rarely) crash: Automated vehicles may encounter unavoidable collisions despite perfect systems, because unpredictable dynamic objects, wildlife, pedestrians, and bicyclists remain hazards.Perfect vehicles may avoid static objects and coordinate with other perfect vehicles, but some road threats remain difficult or impossible to predict.
- Criticism 2: Crashes requiring complex ethical decisions are extremely unlikely: Ethical decisions arise whenever driving involves risk, including ordinary debris avoidance, because vehicles must choose among uncertain consequences and probabilities.The chapter contrasts these uncertain real-world choices with simplified trolley problems having one decision and known outcomes.
- Criticism 1: Automated vehicles will never (or rarely) crash: Automated vehicles will continue interacting with human drivers because fleet replacement is slow and vehicle-only zones are unlikely to cover near-term driving.Even replacing every newly sold U.S. vehicle with an automated one would take 30 years to replace 90% of vehicles.
- Criticism 5: Human drivers will remain responsible: Human availability in level 2 and 3 vehicles does not guarantee ethical oversight because drivers may not maintain attention or take over immediately.In one cited study, 25% of subjects were observed reading during autonomous mode; required warning time is also unspecified for level 3 vehicles.
- Criticism 7: Law will cover ethical situations: Following existing law cannot resolve the full ethics problem because laws lack sufficiently comprehensive, computer-understandable definitions for ambiguous situations.The chapter uses avoiding a tree branch across a double yellow line to illustrate the gap between literal legal compliance and reasonable action.
- Criticism 4: Automated vehicles will never collide with another automated: A safety improvement may still be ethically contested if it reduces overall fatalities while increasing deaths among cyclists or another group.The chapter notes that preliminary evidence does not yet prove automation is safer than human driving.
- Criticism 4: Automated vehicles will never collide with another automated: 1.1 million kilometers without crashing and 482 million kilometers without a fatal crash would be required to establish safety over human drivers with 99% confidence.The chapter states that no automated vehicle had yet safely reached these mileages when written.
3 Relevant Work in Machine Ethics and Moral Modeling
Machine ethics research addresses how autonomous systems can represent and apply moral judgments, while existing approaches reveal difficulties in encoding values, balancing ethical theories, and calibrating models.
- Automated vehicles require ethical responses because society’s values must be articulated across uncertain scenarios and translated into computer-readable language.Research commonly focuses on single choices with known outcomes, whereas vehicle decisions may involve multiple choices and uncertain consequences.
- Deontological systems encode behavioral limits, but literal rules can misinterpret complex situations and may not cover all circumstances.Developing comprehensive rules also requires agreement about human morals, which remains incomplete.
- Utilitarian systems are computationally tractable but require difficult outcome metrics and can disadvantage individuals or groups by prioritizing collective benefit.Property-damage metrics, for example, may favor colliding with a helmeted motorcyclist over a non-helmeted one.
- Automated vehicle ethics will likely require combining multiple ethical theories, including justification, optimization, and individual dose-limit-style constraints.This combination parallels ethical analysis of radiation-risk decisions.
- Existing tools model ethical decisions through utilitarian calculation, duty-based reasoning, casuistry, and expert-calibrated consequence scoring.Examples include Jeremy, W.D., MedEthEx, Truth-Teller, SIROCCO, and the Metric of Evil.
- Computational moral modeling remains in its infancy, with further work needed in model calibration and uncertainty handling.Existing systems nevertheless demonstrate that some ethical problems can be solved automatically.
4 Summary
Automated vehicles will continue to encounter crashes and must autonomously assess and apportion risk because human drivers cannot oversee every decision.
- Automated vehicles must continually assess risk to themselves and others, including whether risk is acceptable and how it should be apportioned.The chapter characterizes these calculations as ethical decisions requiring autonomous choices.
Full Authors’ Information
The chapter identifies Noah J. Goodall as its author, affiliated with the Virginia Center for Transportation Innovation and Research in Charlottesville, Virginia.
- The author is Noah J. Goodall, Ph.D., P.E., of the Virginia Center for Transportation Innovation and Research.The listed address is 530 Edgemont Road, Charlottesville, VA.
Keywords
The chapter concerns automation, autonomous systems, ethics, risk, and morality.
- The keywords are automation, autonomous, ethics, risk, and morality.