Source-linked AI summary
What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
Markus Langer, Daniel Oster, Timo Speith, Holger Hermanns, Lena Kästner, Eva Schmidt, Andreas Sesing, Kevin Baum
TL;DR
XAI research lacks clarity about how explainability approaches should satisfy the varied desiderata of human stakeholders across contexts. This paper reviews stakeholder classes and desiderata, then proposes a conceptual model linking approaches, explanatory information, understanding, context, and desiderata satisfaction to guide interdisciplinary evaluation and development.
Problem
XAI literature often leaves unclear how explainability approaches achieve stakeholders’ desiderata, and more empirical research is needed to identify and investigate those desiderata and their links to approaches.
Method
The paper discusses stakeholder classes, reviews their desiderata, and introduces a conceptual model organizing the concepts and relations involved in evaluating and developing explainability approaches.
Results
The model identifies understanding as a central mediator between explainability approaches and stakeholders’ desiderata satisfaction and highlights relations relevant to interdisciplinary XAI research.
Takeaways & Limitations
The model provides common ground for disciplines involved in XAI and helps guide the evaluation, adjustment, choice, and development of explainability approaches.
Takeaways & Limitations
The paper states that its claims about desiderata relevance require more thorough empirical research and conceptual clarification.
Abstract
from arXiv · showhide
Previous research in Explainable Artificial Intelligence (XAI) suggests that a main aim of explainability approaches is to satisfy specific interests, goals, expectations, needs, and demands regarding artificial systems (we call these stakeholders' desiderata) in a variety of contexts. However, the literature on XAI is vast, spreads out across multiple largely disconnected disciplines, and it often remains unclear how explainability approaches are supposed to achieve the goal of satisfying stakeholders' desiderata. This paper discusses the main classes of stakeholders calling for explainability of artificial systems and reviews their desiderata. We provide a model that explicitly spells out the main concepts and relations necessary to consider and investigate when evaluating, adjusting, choosing, and developing explainability approaches that aim to satisfy stakeholders' desiderata. This model can serve researchers from the variety of different disciplines involved in XAI as a common ground. It emphasizes where there is interdisciplinary potential in the evaluation and the development of explainability approaches.
1. Introduction
XAI is framed as a multidisciplinary effort to make artificial systems understandable to human stakeholders whose varied desiderata motivate explainability. The paper proposes a conceptual model linking explainability approaches, explanatory information, understanding, context, and desiderata satisfaction to guide interdisciplinary research.
- XAI develops approaches that provide explanatory information to help human stakeholders understand artificial systems, their functioning, and their outputs.
- Stakeholders’ desiderata are their interests, goals, expectations, needs, and demands regarding artificial systems, including desires for fairness or trustworthiness.
- Previous XAI research often developed explainability approaches without evaluating whether they satisfy stakeholders’ desiderata, while only a minority of papers evaluated proposed methods.
- The paper argues that evaluation and development should attend to stakeholders’ specific desiderata because explainability success depends on how well those desiderata are satisfied.
- The proposed model treats human understanding as a mediator between explainability approaches and desiderata satisfaction, rather than merely one possible outcome.
- The model organizes concepts and relations to guide evaluating, adjusting, choosing, and developing explainability approaches and to identify where input from other disciplines is needed.
- In the model, explainability approaches provide explanatory information, stakeholders engage with it to facilitate understanding, and context influences these relations and desiderata satisfaction.
2. Stakeholders’ Desiderata
The paper identifies stakeholder classes and their diverse desiderata, then reviews how XAI research relates explainability approaches to those desiderata. It emphasizes clarifying desiderata and conducting systematic empirical research to guide evaluation and development.
- 2.1. Stakeholder Classes: Stakeholders’ desiderata motivate, guide, and affect the explanation process, making their identification and clarification a crucial first step.This applies when evaluating, adjusting, choosing, or developing explainability approaches for a system in context.
- 2.1. Stakeholder Classes: The paper distinguishes five prototypical stakeholder classes: users, developers, affected parties, deployers, and regulators.One person may belong to multiple classes, such as a user affected by a system’s outputs.
- 2.1. Stakeholder Classes: The stakeholder classification is prototypical rather than exhaustive, and finer subclasses may reflect differences in expertise and other factors.The distinction mainly derives from computer science and may neglect other stakeholder classes.
- 2.2. Exemplary Stakeholders’ Desiderata: The literature review extracts desiderata, stakeholder associations, and claims about how XAI findings and outputs can contribute to satisfying them.Sources are categorized by whether they provide no, empirical, mixed, or inconclusive evidence.
- 2.2. Exemplary Stakeholders’ Desiderata: The review presents exemplary desiderata for stakeholder classes while stressing the need for consistent terminology, conceptual clarity, and interdisciplinary discussion.Whether desiderata are justified depends decisively on context and moral and legal considerations.
- 2.2. Exemplary Stakeholders’ Desiderata: Users commonly seek usability and trust, while affected parties centrally seek fairness and morality or ethics.Meaningful information can support decision speed, decision quality, usefulness, and trust calibration; explainability may also improve acceptance.
3. Desiderata Satisfaction requires Understanding
XAI is needed when stakeholders’ desiderata are insufficiently satisfied, with understanding serving as the central link between explanations and desiderata satisfaction. The model distinguishes epistemic and substantial satisfaction and emphasizes that contexts, stakeholders, and competing desiderata shape this relation.
- XAI needs arise when stakeholders’ desiderata are not sufficiently satisfied, motivating analysis of what desiderata satisfaction means.
- Facets of desiderata satisfaction: Each desideratum has epistemic and substantial facets: stakeholders can assess whether a system has required properties, while the system actually possesses them.
- Facets of desiderata satisfaction: Explanations can satisfy epistemic facets of all desiderata, but substantial satisfaction may require system changes beyond explanation alone.
- Understanding: Understanding is pivotal because it provides the basis for assessing systems and can enable substantial desiderata satisfaction without always producing it directly.
- Factors influencing the relation: The relation between understanding and desiderata satisfaction depends on explanatory information, stakeholder characteristics, and context, including whether situations are high or low stakes.
- Interdisciplinary potential: Explainability approaches should therefore be evaluated case by case across stakeholders whose desiderata may conflict and across contextual influences.
4. Understanding Requires Explanatory Information
Explanatory information facilitates understanding, but its effects depend on its characteristics and on stakeholder and contextual factors. Appropriate information must therefore match the purpose, audience, and situation.
- Explanatory information supports understanding by helping stakeholders grasp patterns, infer input-output connections, narrow possible failures, and reduce uncertainty.
- Explanations vary in kind and format, including teleological, mechanistic, causal, text-based, and multimedia information.
- Soundness, completeness, novelty, complexity, and presentation characteristics influence whether explanatory information evokes understanding.
- Explanatory information’s effects vary with its characteristics: people prefer simpler information, and usefulness depends on whether explanations match stakeholders’ goals.
- Teleological explanations are more useful for identifying a phenomenon’s function, whereas mechanistic explanations are more useful for identifying its cause.
- The same information can produce different kinds and degrees of understanding because stakeholder knowledge, beliefs, abilities, desiderata, and context affect interpretation.
- Explanatory information can improve decision-making but may also hamper it when it imposes excessive effort or interacts with human biases.
5. Explanatory Information Requires Explainability Approaches
Explainability approaches provide explanatory information through methods, procedures, and strategies tailored to systems, stakeholders, and contexts. Ante-hoc, post-hoc, interactive, and human-facilitated approaches each have distinct capabilities and limitations.
- Explainability approaches comprise the steps and efforts used to extract and provide explanatory information so stakeholders can understand systems and outputs.
- Ante-hoc and post-hoc approaches: Ante-hoc approaches design systems to be inherently transparent, but they may reduce predictive power and cannot be applied to every system.
- Ante-hoc and post-hoc approaches: Post-hoc approaches extract explanations from models that are usually not inherently transparent and are, in principle, applicable across model types.
- Limitations and adaptations: Both ante-hoc and post-hoc information may remain understandable mainly to expert stakeholders rather than to all intended users.
- Limitations and adaptations: Combining approaches, enabling interaction, or using a human facilitator can provide different information and presentation modes for stakeholder needs.
- Human-system approaches: Human facilitation creates a desiderata hierarchy when one stakeholder’s ability to explain a system is a precondition for another stakeholder’s assessment.
- Scope and model dependence: Approaches differ in local versus global scope and in model-agnostic versus model-specific operation, with model-agnostic methods trading broad applicability for efficiency, accuracy, and explanatory power.
6. Bringing It All Together: Hypothetical Application Scenarios
The model organizes how stakeholders, desiderata, understanding, explanatory information, contexts, and explainability approaches interact. It supports evaluating, adjusting, choosing, and developing approaches for specific purposes and contexts.
- Model and guiding questions: Empirical feedback can reveal which approaches and explanatory information work for particular desiderata and contexts, guiding later improvement.Such feedback may come from stakeholder feedback or empirical investigation.
- Model and guiding questions: The model treats explainability as a process linking explainability approaches, explanatory information, stakeholder understanding, and desiderata satisfaction.It frames these relations as objects of evaluation and development.
- Model and guiding questions: Evaluation asks whether relevant stakeholders and desiderata were identified, whether understanding was acquired, and whether information and presentation fit the context.The model also asks whether the approach provides the right kind and amount of information in the right format.
- Interdisciplinary application: Interdisciplinary collaboration is needed to identify stakeholders, define desiderata, and assess contextual and stakeholder characteristics when applying the model.The paper names law, sociology, psychology, philosophy, computer science, and domain expertise as relevant perspectives.
- Evaluation and discovery scenarios: The framework distinguishes evaluation from discovery: evaluation diagnoses shortcomings, while discovery chooses an existing approach or develops a new one.Discovery also requires determining how desiderata satisfaction should be measured in a specific context.
- Evaluation and discovery scenarios: If understanding is adequate but a desideratum remains substantially unsatisfied, changing the explainability approach may not make that desideratum satisfiable.The paper illustrates this distinction by separating epistemic understanding from substantive fairness of system outputs.
7. Conclusion
The conclusion argues that XAI must address all relevant stakeholders and their desiderata while empirically evaluating explainability approaches. The proposed model supplies a shared conceptual basis for interdisciplinary evaluation and development.
- Scope and motivation: As artificial systems affect more people, the number of stakeholder desiderata continues to grow.The conclusion presents this growth as a motivation for broader XAI analysis.
- Scope and motivation: XAI still needs a comprehensive view of stakeholders and desiderata in socially relevant contexts, together with empirical investigation of desiderata satisfaction.The conclusion identifies both as continuing requirements.
- Contribution: The paper introduces a model that highlights the central concepts and relations through which explainability approaches aim to satisfy stakeholders’ desiderata.The model is presented as the paper’s central contribution.
- Contribution: The model is intended to guide interdisciplinary evaluation and development of explainability approaches concerning stakeholders’ desiderata.The conclusion links this guidance to further advances in XAI research.