Source-linked AI summary
Ethics-Based Auditing to Develop Trustworthy AI
Jakob Mokander, Luciano Floridi
TL;DR
AI safeguards for human decision-making often fail when applied to increasingly pervasive AI systems, creating a need for mechanisms that ensure ethical alignment. The paper examines ethics-based auditing as a governance mechanism, presenting its contributions, continuous-process design, and constraints. It concludes that auditing can strengthen ethical infrastructure and support good governance, but cannot replace ongoing ethical reflection.
Problem
Safeguards for overseeing human decision-making often fail when applied to AI, creating a need for mechanisms that ensure ethical alignment.
Method
The paper examines ethics-based auditing as an independent governance mechanism and argues that audits should continuously monitor and evaluate system outputs while documenting performance characteristics.
Results
Ethics-based auditing can support good governance by strengthening procedural regularity and institutional trust, including decision support, contestability, harm mitigation, accountability, and conflict-of-interest management.
Takeaways & Limitations
Ethics-based auditing affords good governance by strengthening the ethical infrastructure of mature information societies, while individual moral agents must continue ethical reflection.
Takeaways & Limitations
Ethics-based auditing is constrained by conceptual, technical, economic, social, organisational, and institutional factors that must be understood and managed.
Abstract
from arXiv · showhide
A series of recent developments points towards auditing as a promising mechanism to bridge the gap between principles and practice in AI ethics. Building on ongoing discussions concerning ethics-based auditing, we offer three contributions. First, we argue that ethics-based auditing can improve the quality of decision making, increase user satisfaction, unlock growth potential, enable law-making, and relieve human suffering. Second, we highlight current best practices to support the design and implementation of ethics-based auditing: To be feasible and effective, ethics-based auditing should take the form of a continuous and constructive process, approach ethical alignment from a system perspective, and be aligned with public policies and incentives for ethically desirable behaviour. Third, we identify and discuss the constraints associated with ethics-based auditing. Only by understanding and accounting for these constraints can ethics-based auditing facilitate ethical alignment of AI, while enabling society to reap the full economic and social benefits of automation.
1 Towards trustworthy AI
AI ethics is increasingly necessary for good governance, but existing safeguards and voluntary guidelines do not reliably translate principles into actionable oversight. Ethics-based auditing is presented as a promising mechanism to bridge this gap, while requiring realistic expectations about its scope.
- AI’s ethical challenges are becoming a prerequisite concern for good governance, while existing safeguards often fail when applied to AI.
- Ethics guidelines for trustworthy AI remain voluntary, and industry lacks tools and incentives to convert principles into verifiable, actionable criteria.
- Ethics-based auditing is proposed as a mechanism for bridging the gap between AI ethics principles and practice.
- The paper encourages ethics-based auditing while stressing that its achievable outcomes must be assessed realistically.
2 Ethics‑based Auditing of AI—What it is and How it Works
Ethics-based auditing is a structured governance process for assessing whether AI-related behaviour aligns with relevant principles or norms. It can support governance through independent oversight, varied audit approaches, and concrete decision, accountability, and harm-reduction functions.
- Ethics-based auditing assesses an entity’s behaviour for consistency with relevant principles or norms and can influence AI-system behaviour.
- Auditing can identify, visualise, and communicate normative values embedded in systems without attempting to codify ethics.
- Functionality, code, and impact audits respectively examine decision rationale, source code, and the effects of algorithmic outputs.
- Auditing should operate independently of the auditee’s day-to-day management, whether performed by government, contractors, or an internal designated function.
- By promoting procedural regularity and institutional trust, ethics-based auditing supports decision monitoring, contestability, sector-specific governance, harm mitigation, accountability, and conflict-of-interest management.
3 Getting it Right
The paper proposes ethics-based auditing as a continuous, holistic, dialectical, strategic, and design-driven process. Effective auditing monitors systems over time, evaluates them within broader socio-technical contexts, supports questioning, aligns incentives, and feeds back into design.
- The proposed gold standard treats ethics-based auditing as continuous, holistic, dialectical, strategic, and design-driven.
- Audits should continuously monitor and evaluate system outputs while documenting performance characteristics.
- A holistic audit evaluates AI systems as part of larger socio-technical systems and considers available alternatives.
- A dialectical audit ensures that the right ethical questions are asked rather than treating ethics as an answer sheet.
- Strategic value alignment requires auditing frameworks to harmonise with organisational policies and individual incentives.
- Interpretability and robustness should be built in from the start, with auditing providing feedback to continuous system redesign.
4 A Roadmap for Future Research
Ethics-based auditing faces conceptual, technical, economic, social, organisational, and institutional constraints. The paper argues that these constraints must be understood and managed so auditing can contribute to addressing AI’s ethical risks.
- Auditing is not a panacea and is constrained across conceptual, technical, economic, social, organisational, and institutional dimensions.
- Fairness and justice can have incompatible definitions, making prioritisation a political question requiring publicly defensible resolutions.
- AI’s autonomous, complex, and scalable nature can make meaningful quality assurance difficult within test environments.
- These technical constraints arise partly because AI systems can update their internal decision-making logic over time, though future research may transform them.
- Auditing imposes financial and other costs, and power asymmetries may prevent corrective action even when flaws are identified.
- Institutional structures remain unclear, while auditing effectiveness is constrained by tensions between national jurisdictions and globally operating technology.
5 Outlook
Ethics-based auditing is presented as a complement to existing governance tools and as part of holistic AI risk management. Its effectiveness depends on resisting power shifts, preserving government sanctioning authority, and accounting for identified constraints.
- Ethics-based auditing should complement human oversight, certification, and regulation within holistic approaches to managing AI’s ethical risks.
- Governments should retain supreme sanctioning power while independent agencies conduct ethics-based audits.This arrangement is proposed to resist shifting power from juridical courts to private actors without stifling innovation or undermining law-enforcement legitimacy.
- The paper identifies 16 constraints that future research should understand and account for before auditing can address AI’s ethical risks.
- Ethics-based auditing can strengthen the ethical infrastructure of mature information societies and support good governance.It is not intended to replace continuous ethical reflection by individual moral agents.