Source-linked AI summary
The social dilemma of autonomous vehicles
Jean-François Bonnefon, Azim Shariff, Iyad Rahwan
TL;DR
Autonomous vehicles must sometimes choose between protecting passengers and saving more pedestrians, making their moral algorithms difficult to define. Across studies of these choices and regulation, the paper finds support for utilitarian AVs in principle but preference for passenger-protective vehicles personally.
Problem
AVs must sometimes choose between sacrificing passengers and saving more pedestrians, raising questions about which moral algorithms people will accept.
Method
The paper examines moral judgments, passenger-sacrifice choices, and attitudes toward regulating utilitarian AV algorithms.
Results
Participants approved utilitarian AVs that sacrifice passengers to save more lives, including sacrificing one passenger rather than killing ten pedestrians.
Takeaways & Limitations
AV design and regulation must account for both moral approval and public acceptance, including whether policies discourage buyers.
Takeaways & Limitations
The paper acknowledges that its proposed solution faces at least two difficulties.
Abstract
from arXiv · showhide
Autonomous Vehicles (AVs) should reduce traffic accidents, but they will sometimes have to choose between two evils-for example, running over pedestrians or sacrificing itself and its passenger to save them. Defining the algorithms that will help AVs make these moral decisions is a formidable challenge. We found that participants to six MTurk studies approved of utilitarian AVs (that sacrifice their passengers for the greater good), and would like others to buy them, but they would themselves prefer to ride in AVs that protect their passengers at all costs. They would disapprove of enforcing utilitarian AVs, and would be less willing to buy such a regulated AV. Accordingly, regulating for utilitarian algorithms may paradoxically increase casualties by postponing the adoption of a safer technology.
2 Department of Psychology, University of Oregon, Eugene, OR 97403, USA · 3 The Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Autonomous vehicles must encode moral rules for unavoidable-harm scenarios, creating a social dilemma: people endorse utilitarian algorithms for others but prefer self-protective vehicles themselves. Although regulation might enforce safer collective outcomes, participants opposed utilitarian regulation and reported much lower purchase intentions for regulated vehicles.
- 3 The Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA: AVs could eliminate up to 90% of traffic accidents, but unavoidable crashes may require sacrificing pedestrians, passengers, or the vehicle itself.These decisions must be programmed before AVs become a global market because distributing harm is a moral decision.
- 3 The Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA: Manufacturers and regulators must balance consistent moral algorithms, public acceptance, and consumer demand, which may conflict in passenger-sacrifice cases.Utilitarian algorithms minimize casualties but may discourage buyers who believe their own safety should take priority.
- AVs under such a regulation?: The findings identify a social dilemma: people believe utilitarian AVs improve collective outcomes but have personal incentives to ride in vehicles protecting them at all costs.Regulation could therefore delay AV adoption, potentially offsetting lives saved through utilitarian programming with deaths caused by delayed access to broader safety technologies.
- AVs under such a regulation?: Study 5 found that agreement to legally enforce utilitarian sacrifices remained below the midpoint in every scenario, despite higher agreement for algorithms than human drivers (p < 0.002).Agreement was highest when algorithms saved ten lives, with a 95% confidence interval of 33–46.
a b c
This section examines whether autonomous vehicles should sacrifice passengers or pedestrians, including support for legal enforcement and willingness to purchase such vehicles. It compares judgments across scenarios involving strangers, family, and one’s own child.
- Moral and legal judgments: The experiments ask whether AVs will or should sacrifice, including whether sacrifice should be legally enforced.The measures distinguish agreement with legal enforcement from agreement with morality and regulation.
- Personal context and adoption: The studies assess preferences when riding alone, with a coworker, with family, or with one’s own child, as well as preferences for other cars.They also measure willingness to purchase AVs and purchase intentions when the vehicle is not regulated.
- Sacrifice scenarios: Sacrifice judgments vary by target and stakes, including killing a pedestrian to save one or ten people or killing a passenger to save ten.The scenarios separately examine pedestrian sacrifice, passenger sacrifice, and the number of people saved.