PaperlayerView exact revision on arXiv

Source-linked AI summary

Operationalising AI governance through ethics-based auditing: An industry case study

Jakob Mokander, Luciano Floridi

arXiv:2407.06232v1cs.CY

TL;DR

EBA’s feasibility and effectiveness remain insufficiently substantiated by empirical research. This study follows AstraZeneca’s preparation for and experience of an ethics-based AI audit over 12 months. It finds that implementation challenges resemble corporate governance problems, while process audits cannot verify the long-term effects of autonomous and self-learning systems.

  • Problem

    EBA’s feasibility and effectiveness remain insufficiently substantiated by empirical research, despite theoretical justifications for internal and external audits.

    Source: §1 · p. 2

  • Method

    The authors conducted a longitudinal case study, observing AstraZeneca for 12 months as it prepared for and underwent an ethics-based AI audit.

    Source: §1 · p. 3

  • Results

    Large organisations face challenges harmonising standards across decentralised units, defining audit scope, and addressing procurement and external R&D collaborations.

    Source: §8 · pp. 25–26

  • Takeaways & Limitations

    Process audits can assess quality-management claims and risks across the AI life cycle, but cannot verify autonomous and self-learning systems’ effects over time without technical elements.

    Source: §8 · p. 25

    The study describes implementation challenges rather than measuring how EBA procedures affect AI systems’ behaviour.

    Source: §7 · p. 24

Abstract

from arXiv · show

Ethics based auditing (EBA) is a structured process whereby an entitys past or present behaviour is assessed for consistency with moral principles or norms. Recently, EBA has attracted much attention as a governance mechanism that may bridge the gap between principles and practice in AI ethics. However, important aspects of EBA (such as the feasibility and effectiveness of different auditing procedures) have yet to be substantiated by empirical research. In this article, we address this knowledge gap by providing insights from a longitudinal industry case study. Over 12 months, we observed and analysed the internal activities of AstraZeneca, a biopharmaceutical company, as it prepared for and underwent an ethics-based AI audit. While previous literature concerning EBA has focused on proposing evaluation metrics or visualisation techniques, our findings suggest that the main difficulties large multinational organisations face when conducting EBA mirror classical governance challenges. These include ensuring harmonised standards across decentralised organisations, demarcating the scope of the audit, driving internal communication and change management, and measuring actual outcomes. The case study presented in this article contributes to the existing literature by providing a detailed description of the organisational context in which EBA procedures must be integrated to be feasible and effective.

1. Introduction

Ethics-based auditing (EBA) assesses an entity’s behaviour against ethical principles, but its practical feasibility and effectiveness remain insufficiently studied. This paper examines a real-world audit at AstraZeneca and finds that implementation challenges resemble classical corporate governance problems.

1. Introduction

  • EBA assesses an entity’s present or past behaviour against relevant principles or norms, while its feasibility and effectiveness remain under-researched.The authors argue that EBA procedures’ affordances and constraints require evaluation in applied contexts.Source: §1 · pp. 2–3
  • The literature contains few EBA case studies, leaving limited understanding of how organisations implement EBA and what challenges they face.The paper addresses this gap through a longitudinal industry case study.Source: §1 · p. 3
  • Over 12 months, the study observed AstraZeneca as it prepared for and underwent an ethics-based audit, describing governance implementation and challenges that inform transferable EBA practices.The authors frame these contributions as identifying conditions for EBA’s feasibility and effectiveness.Source: §1 · p. 3
  • The case study finds that EBA challenges mirror corporate governance problems, including harmonising standards, defining audit scope and performance indicators, and driving change management.The authors argue that cross-disciplinary knowledge transfer will help develop and implement EBA procedures.Source: §1 · p. 3

2. The need to operationalise AI governance

AI ethics principles alone cannot guarantee ethical design and use, while abstract principles are difficult to translate into verifiable criteria. The section motivates operational governance mechanisms, including ethics-based auditing, alongside other translational tools and legislation.

2. The need to operationalise AI governance

  • AI use can expose people to discrimination, privacy violations, wrongdoing, and reduced self-determination, with health-data processing especially delicate.The cited passage notes risks of reputational damage and suboptimal care for patients.Source: §2 · p. 4
  • Principles alone cannot guarantee ethical AI, and the industry lacks useful tools for translating abstract principles into verifiable criteria.Interpreting concepts such as fairness and justice involves political tensions, and applying principles can require trade-offs.Source: §2 · p. 5
  • Proposed governance tools include impact assessments, model cards, datasheets, human-in-the-loop protocols, standards, reporting guidelines, and software-development requirements.The paper describes these mechanisms as complementary efforts serving corporate AI governance.Source: §2 · p. 5
  • Hard legislation and EBA are complementary because legal requirements cover only some AI systems and ethical scrutiny can extend beyond legality.The passage notes that the EU AI Act applies conformity assessments to specified high-risk systems, while most AI systems are not classified as high-risk.Source: §2 · p. 6

3. AstraZeneca and AI governance

AstraZeneca is a decentralised, R&D-driven biopharmaceutical company with established data, quality, and safety processes and substantial AI use. It published ethical data and AI principles, then paired local implementation with enterprise-wide governance initiatives and an independent audit.

3. AstraZeneca and AI governance

  • The company’s R&D work has long relied on data, quality, and safety management, and its AI applications include drug discovery, medical-image analysis, and adverse-event reporting.The passage describes AI-enabled image analysis as cutting analysis time by over 30% and improving accuracy.Source: §3 · pp. 6–7
  • AstraZeneca’s ethics principles call for data and AI to be private and secure, explainable and transparent, fair, accountable, human-centric, and socially beneficial.The principles were published in November 2020 to help employees and partners navigate AI-related risks.Source: §3 · p. 7
  • AstraZeneca allowed business areas to develop local AI governance structures aligned with published ethics principles, but decentralisation complicated common standards and procedures.Existing governance structures covered areas including quality, data management, corporate social responsibility, sustainability, and product safety.Source: §3 · p. 8
  • Enterprise-wide initiatives included a compliance document, a Responsible AI Playbook, an AI resolution board and consultancy service, and an independent AI audit.The compliance document translated high-level principles into more actionable formulations; the Playbook provided end-to-end guidance and relevant resources.Source: §3 · p. 8

4. An ‘ethics-based’ AI audit

AstraZeneca’s Q4 2021 audit was ethics-based, process-oriented, and conducted with an independent third party. It assessed organisational processes and adherence rather than technically auditing individual AI systems.

4. An ‘ethics-based’ AI audit

  • AstraZeneca’s Q4 2021 audit was ethics-based, process-oriented, and conducted with an independent third party.Its ethics principles provided the baseline for evaluating organisational practices beyond existing legal requirements.Source: §4 · p. 9
  • The audit was external, coordinated by AstraZeneca’s internal audit function and conducted by an independent service provider.The authors note that external experts can help address confirmation bias, though external audits may have reduced access to internal processes.Source: §4 · p. 10
  • The audit combined a high-level review of organisational structures and processes with in-depth audits of specific AI projects.It assessed process soundness and completeness and whether organisational entities adhered to those processes.Source: §4 · pp. 10–11
  • The audit focused on processes and people; it did not review AI systems’ source code or test their outputs.The authors distinguish these technical audits from the process and organisational focus of AstraZeneca’s audit.Source: §4 · p. 11

5. Methodology: An industry case study

The study used a 12-month longitudinal case study of AstraZeneca’s preparation for and conduct of an AI audit, combining participant observation with semi-structured interviews. It examined how industry firms integrate EBA, the challenges they face, and practices for preparing for and conducting audits.

5. Methodology: An industry case study

  • The researchers combined participant observation and semi-structured interviews in a 12-month study of AstraZeneca’s preparation for and conduct of an AI audit.The study adopted a pragmatic industry case-study approach to investigate EBA’s feasibility and effectiveness.Source: §5 · pp. 11–12
  • The research questions addressed how firms integrate EBA into existing governance, the challenges of implementation, and best practices for preparing for and conducting EBA.The study had ethical approval; participants were informed about the research, and no personal details were collected or stored.Source: §5 · pp. 11–13
  • AstraZeneca offered strategic research material because its practical concerns overlapped with the study’s questions and researchers could follow the process from ethics principles to audit evaluation.The observation period covered the principles’ publication in November 2020 through the Q4 2021 audit evaluation. Access was arranged through an existing institutional relationship, and an NDA enabled access to relevant audit meetings.Source: §5 · pp. 11–12
  • Participant observation involved joining weekly meetings, reviewing working documents, and recording audit findings, discussion points, and decisions.The researchers observed internal preparation and evaluation meetings as well as meetings where external auditors questioned employees and reviewed documents; meetings took place online.Source: §5 · p. 12
  • Semi-structured interviews explored themes from audit meetings and stakeholders’ motivations and perspectives; 18 people were interviewed, some more than once.Interviewees were recruited through snowballing, and interviews lasted 1–2 hours; researchers took notes rather than recording them.Source: §5 · p. 12
  • A parallel design ran observation and interviews simultaneously, enabling iterative triangulation of findings and reducing the risk of losing context during qualitative coding.NVivo was used to import, code, and analyse meeting notes.Source: §5 · p. 12

6. Lessons learned from AstraZeneca’s 2021 AI audit

This section draws lessons from AstraZeneca’s 2021 AI audit about the practices and organisational commitments involved in making ethics-based auditing workable. It covers governance across scope, risk, decentralised units, communication and verification, as well as the role of metrics and the staff time auditing requires.

6.1 Balancing legitimate yet competing interests

  • AstraZeneca’s audit found that EBA must balance patient-safety oversight with innovation, define project-specific risk appetite, avoid duplicative controls, and favor continuous risk guidance over post-hoc risk flagging.R&D risks may be necessary for innovation, while excessive red tape could hamper development of potentially lifesaving technologies.Source: §6.1 · p. 13

6.2 Demarcating the material scope for AI governance

  • AstraZeneca used capability examples and risk-proportionate tiers rather than defining AI; this clarified application pragmatically but impeded inventories, leaving auditors reliant on business selection and scope-wide acceptance for verifiability.Examples distinguished ordinary statistical tests from automated tests informing decisions that affect people.Source: §6.2 · pp. 14–15

6.3 Harmonising standards across decentralised organisations

  • Decentralised business areas differed in maturity, structures, and understandings of risk, so compliance-assurance procedures should be harmonised while risk-assurance methods are adapted locally.R&D employees often viewed risk through patient impacts, whereas Commercial employees more often understood it in financial or contractual terms.Source: §6.3 · pp. 15–16

6.4 Internal communication as a key to operationalising AI governance

  • Continuous communication and training were central to operationalising governance; awareness efforts work best with executive support, relevance to daily tasks, and clear explanations of why procedures are needed.Formulating principles took over six months, and embedding them across the business took another year; some employees remained unaware during the audit.Source: §6.4 · pp. 16–17

6.5 Upholding organisational values in procurement and external collaborations

  • Limited visibility into vendors’ training data and processes constrained audits, so EBA should apply harmonised requirements to all systems and add ethics criteria through existing procurement processes.The auditors could examine data practices for in-house systems but lacked full visibility into suppliers’ internal processes and training data.Source: §6.5 · p. 18

6.6 Ethics-based auditing as a catalyst for internal change

  • EBA served purposes beyond risk control, including setting agendas, catalysing organisational change, and expanding unit mandates; the audit prompted communication and governance activity across business areas.Implementation shifted from initial formulation led by the sustainability team to practical implementation by each business area.Source: §6.6 · p. 19

6.7 Making verifiable claims on the basis of ethics-based audits

  • A process-focused audit can assess management systems without reviewing proprietary code or data and support forward-looking collaboration, but cannot verify the long-term impacts of autonomous, self-learning systems.The AstraZeneca audit relied on management representations, interviews, and sample documentation to assess quality-management processes.Source: §6.7 · pp. 19–20

6.8 Measuring progress and demonstrating success

  • EBA metrics should demonstrate tangible improvement against a baseline, but open-ended auditor–staff discussions were more fruitful than binary checklists for probing safeguards and assumptions about system properties.The discussions surfaced what evidence could establish whether systems were fair or robust and challenged assumptions behind concepts such as fairness and transparency.Source: §6.8 · pp. 21–22
  • Metrics should spark ethical deliberation, inform design choices, and visualise embedded normative values rather than decide whether a system is ethical.The authors suggest developing guidance on when and how to use different metrics across the software development lifecycle and holistic EBA procedures.Source: §6.8 · p. 22

6.9 The costs associated with ethics-based audits

  • EBA entails initial preparation and procurement costs as well as variable costs for implementing recommendations, including extra development steps or continuous human oversight.The authors note that the case’s cost figures provide only a ballpark indication, since quantifying governance costs is difficult.Source: §6.9 · pp. 22–23
  • AstraZeneca’s bottom-up ethics-principle process involved consultations and organisation-wide communication and implementation, while approximately four full-time staff coordinated AI-governance implementation in 2020 and 2021.Despite EBA’s costs, businesses have incentives to address vulnerabilities early: fixing a bug during testing can cost up to 15 times more than fixing it during design.Source: §6.9 · pp. 22–23
  • Around 2,000 person-hours went into AstraZeneca’s 14-week Q4 2021 audit, despite its light-touch approach and lack of technical tests on individual AI models.Auditors reviewed the overarching governance structure and selected projects and use cases; employees supplied documents and answered interview questions.Source: §6.9 · p. 23

7. Limitations

The study describes organisational challenges in implementing EBA rather than measuring effects on AI-system behaviour. Its findings are context-specific, with validity, research independence, and generalisability identified as limitations.

7. Limitations

  • Descriptive methods make the link between observations and intended theoretical constructs a key validity concern.The authors identify construct validity as especially relevant to their approach.Source: §7 · pp. 23–24
  • The study observed EBA implementation challenges, not the effects of EBA procedures on AI-system behaviour.The authors cite difficulty assessing specific systems’ ethical risks and focus on describing organisational challenges instead.Source: §7 · p. 24
  • Confirmation bias could lead the findings to replicate previous research, although iterative exchange between literature and case findings was used to manage this risk.The authors also note that longitudinal case studies can strengthen the validity of nonexperimental designs.Source: §7 · p. 24
  • The researcher’s AstraZeneca-funded doctoral studentship raised an independence risk, which the researchers addressed through clear role boundaries and research-ethics practices.The stated risk was pressure to produce results favourable to the industry partner.Source: §7 · p. 24
  • The findings are context-specific rather than neutral, given possible partner bias and limits on disclosure of trade secrets.The researchers triangulated employee input with other sources to reduce the risk that biased input would distort the analysis.Source: §7 · p. 24
  • The authors nevertheless consider the findings relevant to other large corporations integrating EBA into existing governance structures.They point to large firms that have adopted ethics principles and argue that such organisations may face similar challenges.Source: §7 · p. 24

8. Conclusions

The conclusions distinguish what EBA procedures can establish and argue that implementation works best when integrated into organisational governance. The case study identifies governance challenges around harmonisation, scope, procurement, and organisational change.

8. Conclusions

  • Process audits can assess quality-management systems and risks across the AI lifecycle, but without technical elements they cannot verify effects of autonomous, self-learning systems over time.Compared with code audits, process audits require less access to proprietary information and sensitive data.Source: §8 · p. 25
  • EBA is more likely to be effective when integrated into existing governance and communicated through its relevance to employees’ daily work.Siloed or abstractly framed procedures risk being treated as box-ticking; the authors call for auditors to collaborate with managers and developers.Source: §8 · p. 25
  • Decentralised organisations face difficulty harmonising standards because audit requires a predefined baseline, while top-down uniform governance challenges how these organisations are run.The discussion specifically describes large multinational organisations with independently operating business areas.Source: §8 · pp. 25–26
  • Unsettled material scope obstructs EBA: organisations must balance scope precision, ease of application, and generalisability, and agree on what systems and processes are covered.The authors say verifiable claims remain difficult until the scope is accepted across the organisation.Source: §8 · p. 26
  • Unequal treatment of in-house and vendor-developed AI risks shifting risky projects to outsourcing; procurement and collaboration agreements can require alignment with internal governance policies.The authors connect this risk to “ethics dumping” as labelled by Floridi (2019).Source: §8 · p. 26
  • The case study’s challenges mirror classical governance problems, leading the authors to call for management scholars alongside technical, philosophical, and legal researchers in EBA design.The authors note that this conclusion reiterates findings from previous research, including Raisch & Krakowski (2021).Source: §8 · p. 26

APPENDIX 1

The interview guide sought information about participants’ organisational roles and work related to AI. Interviews were open dialogues, with individual emphases adapted to participants’ roles and interests.

APPENDIX 1

  • The interview module sought information from managers, software developers, and internal auditors at AstraZeneca.The listed questions were guides to the module rather than a fixed manuscript.Source: APPENDIX 1 · p. 43
  • Interviews were open dialogues, and their focus varied with each participant’s professional role and personal interests.Source: APPENDIX 1 · p. 43

1. Meta info

The appendix’s interview prompts focused on participants’ roles and work in relation to AI governance. They asked how participants’ responsibilities, initiatives, and daily tasks connected to AI systems and governance development.

1. Meta info

  • Participants were asked about their organisational role, primary responsibilities, and core tasks.Source: §1 · p. 43
  • The guide asked which initiatives participants were executing or planning to address their goals and issues.Source: §1 · p. 43
  • Participants were asked how their daily work related to AI-system design and deployment.Source: §1 · p. 43
  • The guide asked whether participants had helped draft AstraZeneca’s AI ethics principles and internal governance framework.Source: §1 · p. 43

2. Context

The context questions map teams’ AI use and anticipated benefits alongside the technologies, risks, and strengths they associate with AI. They also ask how teams expect to use AI and address related governance factors over the next two years.

2. Context

  • Teams are asked how they define AI systems.Source: §2 · p. 43
  • The questions ask whether teams develop or use AI to support their objectives and what benefits they hope to achieve.They also ask which technologies underpin the applications.Source: §2 · p. 43
  • Teams are asked to assess the probability, sensitivity, and impact of potential system failures and identify AI’s ethical risks and strengths.The ethical risks are considered from organisational and societal perspectives.Source: §2 · p. 43
  • The questions ask how departments expect to use AI in the next two years and which potential risks and governance factors are relevant.Source: §2 · p. 43

3. AstraZeneca AI Ethics principles and governance

The questions examine how AstraZeneca teams manage AI ethical risks, understand the relevance and value of ethics principles, and assign responsibility for governance and quality.

3. AstraZeneca AI Ethics principles and governance

  • Teams are asked which AI and data governance processes, policies, methods, initiatives, and tools manage ethical risks, and how effective they are.Source: §3 · p. 44
  • The questions ask how Data & AI Ethics applies to teams’ AI use, which principles matter, and what value those principles would bring to teams and AstraZeneca.Source: §3 · p. 44
  • Teams are asked who is accountable for AI design or use decisions, how policy compliance is ensured, and whether data and algorithmic quality is measured and assigned to someone.Quality examples include bias, accuracy, and balance.Source: §3 · p. 44

4. Ethics-based auditing

The questions examine the roles, tools, scope, oversight, constraints, and incentives involved in ethics-based auditing, then solicit views on improving its implementation at AstraZeneca.

4. Ethics-based auditing

  • The questions ask whether teams expect auditing to benefit them, what implementation constraints they face, how they manage them, and what incentives or personal accountability mechanisms apply.Source: §4 · p. 44
  • The audit questions ask who participates, what tools and methods are used, what or whom the audit covers, and how it relates to existing accountability and oversight.Source: §4 · p. 44

5. Suggestions for improvements

  • For improvements, respondents are asked how stringent and enforceable audits should be and which ethical requirements they should cover.Source: §5 · p. 45
  • The improvement questions also ask how AI systems should be evaluated and assessed, and what would help auditing succeed within AstraZeneca.Source: §5 · p. 45

Related papers

Loading 2407.06232v1…