PaperlayerView exact revision on arXiv

Source-linked AI summary

Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations

Jakob Mokander, Jessica Morley, Mariarosaria Taddeo, Luciano Floridi

arXiv:2110.10980v1cs.CYcs.SE

TL;DR

Automated decision-making can improve efficiency and enable new solutions, but ethical principles remain difficult to translate into practice. The paper presents ethics-based auditing as a governance mechanism for validating claims about automated decision-making systems through procedural transparency and regularity, while recognising important constraints.

  • Problem

    A gap remains between ethics principles and their practical application to automated decision-making systems, whose broader system effects may be overlooked when isolated malfunctions are examined.

  • Method

    The paper frames ethics-based auditing as a governance mechanism using procedural transparency and regularity to assess automated decision-making systems.

  • Results

    Standardised ethics-based auditing can help organisations validate claims about their automated decision-making systems and demonstrate adherence to specific ethics principles.

  • Takeaways & Limitations

    Responsibility for ensuring that automated decision-making systems are ethically sound remains with the organisations that develop and operate them.

  • Takeaways & Limitations

    Ethics-based audits cannot guarantee a justifiable balance among context-dependent, multi-variable trade-offs.

Abstract

from arXiv · show

Important decisions that impact human lives, livelihoods, and the natural environment are increasingly being automated. Delegating tasks to so-called automated decision-making systems (ADMS) can improve efficiency and enable new solutions. However, these benefits are coupled with ethical challenges. For example, ADMS may produce discriminatory outcomes, violate individual privacy, and undermine human self-determination. New governance mechanisms are thus needed that help organisations design and deploy ADMS in ways that are ethical, while enabling society to reap the full economic and social benefits of automation. In this article, we consider the feasibility and efficacy of ethics-based auditing (EBA) as a governance mechanism that allows organisations to validate claims made about their ADMS. Building on previous work, we define EBA as a structured process whereby an entity's present or past behaviour is assessed for consistency with relevant principles or norms. We then offer three contributions to the existing literature. First, we provide a theoretical explanation of how EBA can contribute to good governance by promoting procedural regularity and transparency. Second, we propose seven criteria for how to design and implement EBA procedures successfully. Third, we identify and discuss the conceptual, technical, social, economic, organisational, and institutional constraints associated with EBA. We conclude that EBA should be considered an integral component of multifaced approaches to managing the ethical risks posed by ADMS.

2 Alan Turing Institute, British Library, 96 Euston Rd, London NW1 2DB

The passage identifies the article and its publication context.

  • The article is titled “Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations.”
  • The article is authored by Mökander, Morley, Taddeo, and colleagues and published in Sci Eng Ethics in 2021.

1. INTRODUCTION

Automated decision-making systems offer efficiency and new solutions but create ethical risks that existing governance mechanisms struggle to address. The paper examines ethics-based auditing as a way to translate principles into practice while acknowledging important limitations.

  • 1.1. Background: ADMS can increase consistency, improve efficiency, and enable solutions to complex problems, but may also produce bias, discrimination, privacy violations, and reduced self-determination.
  • 1.1. Background: Traditional governance mechanisms often fail when applied to ADMS because automated delegation curtails ethical deliberation and shifts attention toward system design and deployment.
  • 1.2. From principles to practice: Ethics guidelines remain difficult to operationalise because principles can be interpreted too rigidly or too flexibly, leaving a gap between ethical principles and practice.
  • 1.2. From principles to practice: The paper investigates ethics-based auditing as a governance mechanism for operationalising ethical commitments and validating claims about ADMS.
  • 1.3. Scope, limitations, and outline: The article focuses on ethical alignment beyond legal compliance, excludes legal auditing and normative-ethics reviews, and notes that EBA does not itself guarantee morally good outcomes.
  • 1.3. Scope, limitations, and outline: The authors contribute a theoretical account of EBA, seven implementation criteria, and an analysis of its conceptual, technical, economic, social, organisational, and institutional constraints.

2. AUTOMATED DECISION-MAKING SYSTEMS

ADMS are sociotechnical systems whose autonomy, complexity, and scalability create distinct governance challenges. These features complicate accountability, causal attribution, and the management of system-wide effects.

  • 2. AUTOMATED DECISION-MAKING SYSTEMS: ADMS are defined as sociotechnical systems encompassing a decision model, computable algorithm, input data, and the surrounding use environment.
  • 2. AUTOMATED DECISION-MAKING SYSTEMS: Autonomy makes it difficult to predict ADMS behaviour and assign accountability when harms occur, especially as systems adapt or act through networks of human and non-human agents.
  • 2. AUTOMATED DECISION-MAKING SYSTEMS: Technical complexity and opacity can hinder links between outcomes and causes, undermining accountability through mechanisms such as hidden layers in neural networks.
  • 2. AUTOMATED DECISION-MAKING SYSTEMS: Scalability makes externalities harder to manage because effects may be difficult to predict, cross borders and generations, and complicate agreement on legitimate values.
  • 2. AUTOMATED DECISION-MAKING SYSTEMS: The fallacy of the broken part warns that serious ADMS failures may reflect requirement flaws rather than coding errors, so ethical audits must examine impacts and stakeholder goals beyond source code.

3. ETHICS-BASED AUDITING

EBA is a structured, purpose-oriented governance process that assesses ADMS against predefined ethical principles, verifies organisational claims, and links auditing to existing governance structures.

  • EBA assesses an entity’s present or past behaviour for consistency with relevant principles or norms.
  • EBA governs ADMS across conceptualisation, design, deployment, and use, connecting guiding principles with managerial intervention.
  • Functionality, code, and impact audits examine decision rationales, source code, and the types, severity, and prevalence of system effects.
  • EBA informs, interlinks, and assesses governance structures, providing a basis for—but not reducible to—certification.
  • EBA verifies claims through purpose-oriented methods and tools, including stakeholder consultation, adversarial testing, software, and standardised reporting formats.
  • Independent auditing separates auditors from the regular chain of command to reduce collusion risks and clarify responsibility for harms or failures.

4. STATUS QUO: EXISTING EBA FRAMEWORKS AND TOOLS

Existing EBA frameworks and tools span impact-assessment procedures, governance strategies, checklists, visualisations, and explanation methods. Together, they support detection, evaluation, and communication of ethical risks, but often address only selected concerns or are used in isolation.

  • Top-down governance strategies and bottom-up data-regulation frameworks provide normative guidelines and procedural blueprints for EBA.The latter include contextualisation of data processing and independent evaluation of software procured from third-party vendors.
  • Data-regulation frameworks often target specific ethical concerns, such as privacy, so focusing on one or a few values may cause holistic sub-optimisation.
  • Reviewed EBA frameworks converge around impact assessments that describe systems, define criteria, disclose data use, assess impacts, evaluate benefits and risks, and create feedback loops.IAF summarises this procedure in eight steps, including reliability, safety, transparency, documentation, and periodic evaluation.
  • EBA tools measure, evaluate, or visualise system properties through fairness analytics, model comparisons, output visualisation, and interpretability techniques.
  • Explanation tools may fail to reveal unjustified protected-feature effects when proxy variables mediate decisions.
  • Local interpretability should target individual stakeholders and specific purposes such as internal governance, reputation management, or third-party verification.

5. A VISION FOR ETHICS-BASED AUDITING OF ADMS

The paper presents EBA as a structured process that combines frameworks and tools across the software lifecycle to identify, communicate, and address ethical risks. Its potential benefits include better oversight, contextual alignment, harm prevention, and accountability, but these benefits are not guaranteed.

  • EBA should combine conceptual frameworks and software tools into a structured process monitoring each development stage for potential ethical failures.The proposed process integrates functionality, code, and impact auditing.
  • Continuous assessment feeds information about ADMS ethical behaviour into subsequent design and application iterations.
  • EBA can support executives and legislators by defining and monitoring outcomes, while helping developers improve performance and establish safeguards against unwanted behaviour.
  • EBA can increase trust and user satisfaction through operational consistency, procedural transparency, and actionable explanations that support contestation.
  • EBA enables contextual alignment by allowing organisations to select relevant ethics principles and publicly justify those choices.
  • EBA can help anticipate harms, balance explanation with privacy and intellectual-property rights, and clarify accountability across stakeholders.
  • The methodological advantages are potential rather than guaranteed and depend on contextual factors and framework design.

6. CRITERIA FOR SUCCESSFUL IMPLEMENTATION

The paper proposes seven aspirational criteria for feasible and effective EBA: holistic, traceable, accountable, strategic, dialectic, continuous, and driving redesign. Their implementation remains constrained by organisational, contextual, and adversarial factors.

  • EBA procedures should be holistic, treating ADMS as integrated components of larger sociotechnical contexts.
  • EBA should be traceable by assigning responsibilities and documenting decisions so organisations can verify claims and follow up.
  • EBA should be accountable by linking unethical behaviour to proportional sanctions and strengthening interpersonal accountability.
  • EBA should be strategic, aligning ethical values with policies, organisational strategies, incentives, and individual responsibilities.
  • EBA should be dialectic: auditors and system owners collaboratively develop context-specific methods rather than follow a fixed answer sheet.
  • EBA should be continuous, monitoring, evaluating, and communicating system impacts over time, including unexpected or harmful behaviour.
  • EBA should drive redesign by feeding audit results back into the ongoing development and modification of ADMS.
  • The seven criteria are aspirational and unlikely to be satisfied simultaneously, while best practices for EBA have yet to emerge.

7. DISCUSSION: CONSTRAINTS ASSOCIATED WITH ETHICS-BASED AUDITING

EBA faces conceptual, technical, economic, social, organisational, and institutional constraints that limit what audits can establish and how fairly they can operate. These constraints require trade-offs, continuous management, and attention to power, costs, and real-world contexts.

  • Overview: EBA is subject to conceptual, technical, social, economic, organisational, and institutional constraints, even when implemented according to proposed criteria.The authors argue that these constraints must be understood to design feasible and effective procedures.
  • Conceptual constraints: Conceptual constraints arise because ethical auditing must address trade-offs between legitimate but conflicting values such as privacy, service personalisation, convenience, and autonomy.Audits cannot guarantee that a justifiable balance has been reached, but they can identify, evaluate, and communicate the trade-offs.
  • Conceptual constraints: EBA can make implicit choices and tensions visible, give stakeholders a voice, and support publicly defensible resolutions, but prioritising conflicting ethical principles remains normative.Different fairness definitions may be mutually exclusive or may increase discrimination according to other interpretations.
  • Technical constraints: Technical constraints stem from ADMS complexity, opacity, agility, scalability, data dependence, and differences between controlled test environments and real-life settings.These factors hinder responsibility tracing, make auditing potentially perturbative, and limit inference about later performance; continuous auditing and live experimentation are proposed responses.
  • Technical constraints: EBA may fail to assure compliance for systems not yet developed, and most current schemes do not protect user data from third-party auditors.Simulation can be complemented by continuous auditing of live applications, but meaningful quality assurance remains unavailable in some test environments.
  • Economic, social, and institutional constraints: Economic and institutional constraints include unequal costs and benefits, weak incentives, adversarial behaviour, power asymmetries, and the absence of a clear institutional structure.The paper warns that audits may burden smaller companies, reinforce private power, and require regulators to retain sanctioning authority.

8. CONCLUSIONS

The paper concludes that EBA can validate ADMS claims, strengthen procedural regularity and institutional trust, and complement oversight, certification, and regulation. However, it is necessary but insufficient: continuous ethical deliberation, regulatory authority, and broader societal debate remain essential.

  • The responsibility for ensuring that ADMS are ethically sound lies with the organisations that develop and operate them.
  • EBA can address some ethical challenges posed by autonomous, complex, and scalable ADMS, but it is necessary and not sufficient for ethical alignment.It cannot anticipate all long-term and indirect consequences or resolve conflicts between irreconcilable normative values.
  • Regulators should retain supreme sanctioning power and authorise independent agencies to conduct EBA rather than transferring ultimate responsibility from courts to private actors.
  • EBA frameworks should remain part of broader debate about the kind of society people want and the moral compromises they accept.The paper presents EBA as a component of multifaceted approaches, not a substitute for continuous ethical reflection and deliberation.
  • EBA helps organisations validate claims about ADMS and contributes to procedural regularity, transparency, and institutional trust.It is intended to complement, enhance, and interlink human oversight, certification, and regulation.

Appendix A – List of reviewed EBA frameworks and tools

The appendix documents the EBA frameworks and tools reviewed and the systematic literature-review process used to identify generalisable lessons. The review included 122 articles and reports and evaluated selected recent, relevant tools and frameworks.

  • Table 1 lists reviewed EBA frameworks and tools, with the methodology for producing the table described in Appendix B.
  • The study’s purpose was to improve understanding of EBA and how it can help organisations develop and deploy ethically sound ADMS in practice.
  • The research addressed available EBA tools, EBA’s contribution to mitigating ADMS ethical risks, and conceptual, technical, economic, social, organisational, and institutional constraints.
  • The literature review searched five databases using terms covering auditing, ethics, fairness, transparency, automated decision-making, artificial intelligence, and algorithms.The review focused on common ADMS features from which ethical challenges arise.
  • 122 articles and reports were included in the systematised literature review.Existing auditing tools and frameworks were then reviewed to extract generalisable lessons about implementation opportunities and constraints.
  • The reviewed tools and frameworks were selected because they were recent, relevant, and developed by reputable organisations.

Appendix C – Summary table of constraints associated with EBA of ADMS

The summary table presents conceptual, technical, social, economic, organisational, and institutional constraints associated with ethics-based auditing of automated decision-making systems. Understanding and accounting for these constraints is framed as necessary for feasible, effective procedures and as a basis for future research and policymaking.

  • EBA of ADMS is subject to conceptual, technical, social, economic, organisational, and institutional constraints.These constraints are summarised in Table 2.
  • Understanding and accounting for these constraints is necessary to design feasible and effective auditing procedures.
  • The listed constraints are intended to provide a roadmap for future research.
  • The constraints are also intended to guide policymakers’ attempts to support emerging EBA.
  • The summary table was first published in a short commentary article by Mökander and Floridi in 2021.
Loading 2110.10980v1…