Source-linked AI summary
Building Ethics into Artificial Intelligence
Han Yu, Zhiqi Shen, Chunyan Miao, Cyril Leung, Victor R. Lesser, Qiang Yang
TL;DR
AI research needs stronger technical approaches for incorporating ethics into systems that increasingly interact with humans, beyond surveys focused mainly on psychological, social, and legal issues. This paper surveys recent technical work from leading AI venues, organizes it into four areas, and identifies current patterns and future research needs. The literature emphasizes individual ethical frameworks, while collective decision mechanisms, human-AI interaction ethics, preference representation, cultural data, and explainability remain important directions.
Problem
Technical approaches to ethical AI decision-making remain insufficiently reviewed, while existing surveys largely focus on psychological, social, and legal aspects.
Method
The paper surveys recent technical advances from leading AI conferences and journals and organizes them into four areas of AI governance.
Results
Recent work focuses mainly on generalizable individual ethical decision frameworks combining rule-based and example-based approaches, alongside emerging collective and human-AI interaction research.
Takeaways & Limitations
Further progress requires culturally diverse ethical-dilemma data, improved mechanisms for representing collective preferences, interdisciplinary engagement, and explainable ethical decisions.
Takeaways & Limitations
Self-reported preferences in crowdsourced ethical-dilemma data can deviate from actual choice behavior, limiting their direct reflection of decisions.
Abstract
from arXiv · showhide
As artificial intelligence (AI) systems become increasingly ubiquitous, the topic of AI governance for ethical decision-making by AI has captured public imagination. Within the AI research community, this topic remains less familiar to many researchers. In this paper, we complement existing surveys, which largely focused on the psychological, social and legal discussions of the topic, with an analysis of recent advances in technical solutions for AI governance. By reviewing publications in leading AI conferences including AAAI, AAMAS, ECAI and IJCAI, we propose a taxonomy which divides the field into four areas: 1) exploring ethical dilemmas; 2) individual ethical decision frameworks; 3) collective ethical decision frameworks; and 4) ethics in human-AI interactions. We highlight the intuitions and key techniques used in each approach, and discuss promising future research directions towards successful integration of ethical AI systems into human societies.
1 Introduction
Ethical decision-making has become an important concern as AI systems increasingly interact with people, while technical research on implementing ethics remains comparatively underreviewed. The paper addresses this gap by surveying recent technical advances and organizing them into four areas.
- Existing autonomous systems already warrant serious attention to incorporating ethical considerations.
- Existing AI-governance surveys largely emphasize psychological, social, and legal challenges rather than technical implementation.
- The paper surveys recent techniques from leading AI conferences and journals to address this gap.
- The proposed taxonomy divides AI governance techniques into exploring dilemmas, individual frameworks, collective frameworks, and human-AI interactions.
- The paper also discusses future research directions for integrating ethical AI systems into human societies.
2 Exploring Ethical Dilemmas
Ethical-dilemma exploration helps identify human preferences and principles before ethical behavior is encoded into AI systems. The reviewed approaches use expert review and crowdsourcing, but reported preferences may not match actual behavior.
- Ethical-dilemma exploration is presented as a first step toward building AI systems that behave ethically.
- GenEth uses expert review, ethicist participation, and representation schemas to codify ethical principles for application domains.
- Moral Machine crowdsources judgments about autonomous-vehicle dilemmas across considerations including lives saved, passengers, law, age, and social value.
- 3 million participants contributed feedback indicating a general preference for sacrificing an AV when more lives can be saved.
- Self-reported preferences may diverge from actual behavior, leaving the extent to which these findings reflect real choices unresolved.
3 Individual Ethical Decision Frameworks
Individual ethical decision frameworks provide generalizable mechanisms for agents to evaluate their own or others’ actions. The reviewed approaches combine rules, examples, formal representations, reasoning, and learned ethical values.
- Generalizable frameworks are preferred over ad-hoc rules because ethical bounds can be contextual and difficult to define at design time.
- MoralDM combines first-principles reasoning over ethical rules with analogical reasoning from previously resolved cases.
- A BDI-based framework represents ethical theories and uses awareness and evaluation processes to judge agents’ actions.
- Game-theoretic approaches can represent dilemmas with extensive forms extended by passive actions to account for protected values.
- CP-nets can represent exogenous ethical priorities alongside endogenous subjective preferences and compare their distances.
- Other approaches shift ethical reasoning toward agents through action languages, answer set programming, or ethics shaping of reinforcement-learning rewards.
4 Collective Ethical Decision Frameworks
Collective ethical decision frameworks govern groups of autonomous entities through social norms, role-based structures, preference aggregation, and voting. Their development is constrained by the difficulty of representing numerous, dependent, uncertain, or imprecise ethical preferences.
- Collective ethical decision-making is motivated by the view that individually ethical agents may still need rules governing social norms.
- Distributed social-norm frameworks preserve individual autonomy while imposing collective roles, privileges, and penalties.
- Human-agent collectives can assign agents different ethical roles and aggregate their constrained evaluations through preference aggregation or multi-agent voting.
- Collective ethical decision-making needs new preference representations because candidate actions may vastly outnumber agents and may be dependent, missing, or imprecise.
- A voting-based system learns individual preference models from Moral Machine data and summarizes them into an approximate collective model.
5 Ethics in Human-AI Interactions
Ethics in human-AI interactions concerns protecting autonomy, balancing benefits and risks fairly, and understanding how agents influence or respond to people. Studies of persuasion and emotional responses show that ethical interaction depends on both strategy and agent behavior.
- Ethics in Human-AI Interactions: Human-influencing AI should preserve personal autonomy, ensure benefits outweigh risks, and distribute benefits and risks fairly.The Belmont principles emphasize free will, favorable benefit-risk balance, and nondiscrimination based on personal backgrounds.
- Ethics in Human-AI Interactions: Persuasion-agent studies used emotional appeals, utilitarian arguments, and lying in a trolley dilemma involving harm to save five people.The strategies were delivered either by a human authority or by an AI persuasion agent.
- Ethics in Human-AI Interactions: Participants held a strong preconceived negative attitude toward persuasion agents, while argumentation-based and lying-based strategies outperformed emotional persuasion.The comparison was conducted in an ethical dilemma requiring violation of the value that one should not kill.
- Ethics in Human-AI Interactions: Ethically appropriate emotional responses can enhance human-AI interaction even when emotional appeals are ineffective for persuasion under ethical dilemmas.A Coping Theory-based approach lets agents respond to strong negative emotions by changing their appraisal of the situation.
6 Discussions
The surveyed technical work concentrates on generalizable individual ethical decision frameworks, while collective decision-making and human-AI interaction remain active areas requiring further development. The discussion also emphasizes broader expertise, regulation, and ethics education.
- 6 Discussions: Most surveyed work develops generalizable individual ethical decision frameworks that combine rule-based and example-based approaches.These frameworks aim to resolve ethical dilemmas using learned examples alongside explicit rules.
- 6 Discussions: Collective ethical decision-making has used multi-agent voting, but mechanisms for representing agents’ ethical preferences remain underdeveloped.The limitation concerns how ethical preferences are represented before collective decisions are reached.
- 6 Discussions: Human-AI interaction research currently focuses on ethical recommendations and affective expression of AI ethical judgments.The discussion identifies both recommendation behavior and emotional expression as interaction-level concerns.
- 6 Discussions: The paper calls for a global and unified AI regulatory framework as autonomous vehicles, autonomous weapons, and cryptocurrencies affect societies.The proposed regulatory concern covers emerging AI-related technologies and their ethical issues.
- 6 Discussions: Ethics education and collaboration with ethics and decision-making communities are presented as needs for developing ethical AI technologies.The paper notes that consequentialist ethics is more familiar to AI researchers than deontological and virtue ethics.
7 Future Research Directions
Future research should improve how human ethical preferences are measured, adapt governance to changing social contexts, and make AI decisions explainable. It must also account for strategic changes in human behavior when systems are known to follow ethical principles.
- 7 Future Research Directions: Self-reported preferences in crowdsourced ethical-dilemma studies can diverge from actual choice behavior.The paper proposes social-systems analysis and possible transfer learning to model ethical differences across cultural and other settings.
- 7 Future Research Directions: Future social-contract research should address responsibility, monitoring, and enforcement as AI becomes increasingly ubiquitous.The paper describes this work as dynamic and interdisciplinary because relevant cultural, social, legal, philosophical, and technological realities change.
- 7 Future Research Directions: Argumentation-based explainable AI is proposed as a starting point for explaining AI decisions under human ethics.The paper warns that explanation detail requires trade-offs: full transparency may overwhelm time-critical users, while insufficient transparency may hamper trust.
- 7 Future Research Directions: Ethically constrained AI may alter human behavior in ways that undermine the systems’ design objectives.The paper illustrates this possibility with an ethical autonomous gun that could be disabled by a child regarded as a non-combatant.