Source-linked AI summary
Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research
AKM Bahalul Haque, A. K. M. Najmul Islam, Patrick Mikalef
TL;DR
XAI research needs synthesis from the end user’s perspective because opaque AI systems make decision procedures difficult to understand. The paper conducts an SLR of selected literature, identifies explanation needs and XAI effects, and develops a framework with future research avenues. Its review is limited by considering empirical studies and searching only Scopus and Web of Science.
Problem
Opaque AI systems provide insufficient information about how conclusions are derived, motivating research on explainability for end users.
Method
The paper conducts an SLR using Scopus and Web of Science searches, citation chaining, and inclusion and exclusion criteria to select relevant publications.
Results
The review identifies four dimensions of explanation needs, five XAI effects, ten application domains, future research avenues, and a comprehensive end-user framework.
Takeaways & Limitations
The framework organizes XAI explanation needs and effects on end users while framing future research as research questions and possible paths.
Takeaways & Limitations
The study considers only empirical XAI studies and searches Scopus and Web of Science, so important theoretical or database-indexed studies may be missed.
Abstract
from arXiv · showhide
The final search query for the Systematic Literature Review (SLR) was conducted on 15th July 2022. Initially, we extracted 1707 journal and conference articles from the Scopus and Web of Science databases. Inclusion and exclusion criteria were then applied, and 58 articles were selected for the SLR. The findings show four dimensions that shape the AI explanation, which are format (explanation representation format), completeness (explanation should contain all required information, including the supplementary information), accuracy (information regarding the accuracy of the explanation), and currency (explanation should contain recent information). Moreover, along with the automatic representation of the explanation, the users can request additional information if needed. We have also found five dimensions of XAI effects: trust, transparency, understandability, usability, and fairness. In addition, we investigated current knowledge from selected articles to problematize future research agendas as research questions along with possible research paths. Consequently, a comprehensive framework of XAI and its possible effects on user behavior has been developed.
1. Introduction
As AI-based decision-making becomes widespread, opaque models create challenges for end-user trust and effective use. This paper synthesizes XAI research from the end user’s perspective and develops a framework linking explanation needs with XAI effects.
- Opaque AI models do not reveal enough information about how conclusions are derived, which can reduce end users’ trust.
- Understanding AI decision procedures is crucial for effective decisions, particularly in mission-critical contexts such as healthcare.
- Automated government decisions, including asylum and residence applications, create sensitive settings where explainability is needed by users and decision participants.
- The review identifies end-user explanation needs, XAI effects, research gaps, future directions, and a comprehensive framework.
2. Background
The background distinguishes XAI-related concepts and reviews prior systematic studies across ethical, human-centric, personalized, behavioral, and healthcare perspectives.
- Explainability presents machine-model decisions in human-understandable form, while interpretability explains how or why a model produced a particular prediction.
- Transparency concerns whether a model’s steps can be explained simply, whereas understandability concerns whether users can recognize its features without knowing its inner composition.
- Prior XAI reviews address ethical risks of black-box systems, human-centric design patterns, personalized explanations, behavioral interactions, and healthcare applications.
- Existing research has emphasized policy summarization, human collaboration, visualization, and verification, while gaps remain in customization, user testing, and scalability.
3. Methodology
The study uses a systematic literature review of XAI research, searching Scopus and Web of Science through July 15, 2022 and filtering studies with explicit criteria.
- The authors conducted an SLR using Boolean search strings in the Scopus and Web of Science databases, with the final search on July 15, 2022.
- Inclusion and exclusion criteria were defined to filter irrelevant studies and establish the final article list.
- 1707 journal and conference articles remained after duplicate removal; title and abstract screening removed 1190 articles, leaving 517 for full-text assessment.
- Articles without empirical studies were excluded from the literature selection.
4. Research trend
The selected XAI literature comprised 58 studies, predominantly conference papers, and publication activity increased from 2018 onward.
- 58 studies were included in the SLR, comprising 13 journal articles and 45 conference articles.
- XAI publications increased from 2018 onward, indicating growing research interest in recent years.
5. Synthesis of prior literature
This section critically analyzes the selected research studies and summarizes their findings through a synthesis of prior literature. It covers current knowledge representation and research domains.
- The analysis is organized into current knowledge representation and research domains.
- Table 7 presents the synthesis of prior literature.
- The section provides a critical overview of findings from the selected research studies.
5.1. Current knowledge representations
Prior literature describes XAI explanations through representation and information-quality dimensions, while linking them to user effects such as trust, transparency, understandability, usability, and fairness. The review also identifies measurement and development gaps, including limited empirical evidence on explanation quality and stakeholder-focused guidelines.
- Explanation dimensions: Four explanation dimensions are format, completeness, accuracy, and currency.Format concerns representation mode; completeness includes required and on-demand supplementary information; currency concerns recent information.
- Format: XAI explanations may be textual, visual, auditory, or hybrid, with formats varying across application domains.Healthcare examples require textual and visual formats, while virtual assistants may combine voice, text, and visual explanations.
- Completeness: Complete explanations provide required domain information and supplementary details that users can request when needed.Healthcare examples include demographic information, symptoms, test data, and initial evaluations; contextual references can also be supplied on request.
- Accuracy: Users’ perceptions of explanation accuracy are shaped by personalized prioritization, counterfactual information, supplementary data, confidence values, and sequential procedures.These elements can help users verify decisions and accept or ignore system recommendations.
- Currency: Currency combines automatic explanations with on-demand access to recent, historical, contextual, and user-requirement information.Up-to-date information is emphasized for fraud detection, loan approval, recruitment, and other mission-critical decisions.
- XAI effects: The review links explanation dimensions to five XAI effects: trust, transparency, understandability, usability, and fairness.Trust is associated with stated and observed model accuracy, contextual information, historical data, references, confidence, and explanation style.
- Future research gaps: The literature lacks comprehensive XAI development standards and empirical measurement of explanation quality and stakeholder impact.Proposed research paths include identifying stakeholders, codesigning with them, developing measurement scales, and collecting user responses.
- XAI effects: Prior research reports that transparency and understandability depend on revealing decision procedures, user knowledge, sequential interactions, logical reasoning, contextual information, and counterfactuals.Specific explanation styles and accessible, interactive interfaces are also described as improving usability for nontechnical stakeholders.
5.2. Research domains
The review identifies ten application domains for XAI and shows that explanation needs vary by domain, user, and decision context. Across these settings, users commonly need tailored combinations of automatic and on-demand explanations.
- XAI has been used in 10 domains, including healthcare, media and entertainment, education, transportation, finance, e-commerce, human resource management, digital assistants, e-governance, and social networking.
- Healthcare: Healthcare research focuses on clinical decision making, disease diagnosis, and health-related recommendation systems, primarily for doctors with limited technical knowledge.
- Media and entertainment; Education: Media and entertainment users prefer personalized recommendations presented in various explanation styles, while education studies report improved system usability from explanations.
- Transportation: Transportation users may require case-based, visual, textual, light-indicator, hybrid, or on-demand explanations depending on the system and decision context.
- Finance and other applications: Financial, e-commerce, human-resource, digital-assistant, criminal-justice, and social-networking applications require explanations addressing decision details, transparency, interaction, fairness, or system behavior.
- XAI development: XAI development guidelines involve designers, developers, domain experts, product managers, data scientists, auditors, and end users, but comprehensive standards remain limited.
6. Critical analysis of future research agendas
The paper problematizes future XAI research by identifying gaps in standardization, explanation representation, measurement, stakeholder communication, and longitudinal user research. It translates these gaps into research questions and possible research paths.
- Future research agenda: Future XAI research is organized into standardization, representation, and human-effects themes, with research questions and feasible paths proposed for unexplored areas.The analysis critically examines methodological, conceptual, and development issues rather than merely listing gaps.
- XAI standardization: Existing XAI studies lack comprehensive development guidelines or standards, despite proposals for UI guidelines and requirement-elicitation question banks.The review identifies standardization as a central research need and suggests stakeholder involvement, domain-specific guidance, and design science research.
- Regulation and compliance: Empirical evidence is limited on regulatory and compliance aspects of XAI, motivating co-design with regulators, auditors, privacy officers, and other stakeholders.A data protection impact assessment is also identified as a crucial research step.
- Explanation quality: The literature does not empirically measure explanation quality dimensions, creating a need to adapt or develop scales for format, completeness, accuracy, and currency.The review distinguishes explanation quality dimensions from information quality dimensions.
- XAI visualization: No reviewed article investigates how to represent explanations to low-literate people or measure their perceptions, supporting experiments that evaluate representations and validate measurement scales.Suggested work includes identifying low-literate groups and collecting responses to AI decisions and explanations.
- XAI effects: Because most prior XAI research is cross-sectional, longitudinal designs could examine changes in user perceptions at group and individual levels over time.The paper recommends developing research models and testing them longitudinally.
7. Synthesized framework for XAI research from users’ perspectives
The synthesized framework links explanation-quality characteristics and explanation timing to user-oriented effects and subsequent AI-related behavior. It proposes direct relationships between object-based and behavioral beliefs.
- Object-based beliefs: Explanation quality is represented by format, completeness, accuracy, and currency, alongside automatic and on-demand explanation timing.These are treated as object-based beliefs describing characteristics of the technology.
- Behavioral beliefs: The framework links explanation-related factors to trust, transparency, understandability, usability, and fairness.These factors are treated as behavioral beliefs concerning anticipated consequences of technology use.
- Behavioral intention: Behavioral beliefs in the framework are proposed to influence behavioral intention, including AI adoption and AI use.The framework is presented graphically in Fig. 2.
- Framework relationships: Unlike the mediated relationships described in the original conceptualization, the framework proposes direct relationships between object-based and behavioral beliefs.This proposal is grounded in a recent empirical study cited by the paper.
8. Implications
The paper contributes a user-centered synthesis of XAI concepts, effects, future research directions, and a framework connecting explanation factors with behavior. Practically, it offers guidance for designing and evaluating human-centric AI across domains.
- Theoretical implications: The SLR identifies end users’ explanation needs and their impacts, extending prior XAI literature reviews.This contribution is positioned as one of the few broad investigations of AI end users’ explanation needs.
- Theoretical implications: The paper conceptualizes AI explanation quality through format, completeness, accuracy, and currency, while also identifying automatic and on-demand explanation.The conceptualization adapts Wixom and Todd’s information-quality dimensions.
- Theoretical implications: The SLR links five XAI effects—trust, transparency, understandability, usability, and fairness—to XAI representation dimensions.These effects are positioned as the most important effects identified in the reviewed literature.
- Theoretical implications: Three future-research themes and nine research questions are proposed, together with possible ways to address them.The themes are XAI standardization, XAI visualization, and XAI effects.
- Theoretical implications: The framework connects explanation-related factors and XAI effects, proposing that these effects can influence AI adoption and use.It also supports development and testing of research models.
- Practical implications: The SLR can guide human-centric AI design and measurement of consequences across mission-critical, industrial, and corporate decision-making contexts.Designers are encouraged to support both automatic and on-demand explanations and address identified explanation-quality dimensions.
9. Conclusion
This review synthesizes XAI from end users’ perspectives, identifying explanation-quality dimensions, user effects, and future research avenues in a comprehensive framework. Its conclusions are bounded by the empirical-study focus, database selection, and adopted conceptual dimensions.
- Conclusion: The review identifies explanation-quality dimensions and links them to effects on trust, understandability, fairness, and transparency.It also proposes that these effects can motivate users to adopt and use AI-based systems.
- Limitations: The study reviewed only empirical XAI research, so future reviews could include theoretical papers.This limitation narrows the type of evidence included in the synthesis.
- Limitations: Searching Scopus and Web of Science may have excluded important studies, which future work could seek through additional databases.The limitation concerns the review’s database coverage.
- Limitations: Using Wixom and Todd’s information-quality dimensions may have omitted additional explanation-quality dimensions proposed by other researchers.Future studies could apply alternative information-quality dimensions.
CRediT authorship contribution statement
The contribution statement assigns distinct roles across conceptualization, review and analysis, search and data work, writing, supervision, and reviewer-response activities.
- AKM Bahalul Haque handled conceptualization, methodology, the primary search, data collection, original-draft writing, and reviewer-comment responses.
- A.K.M. Najmul Islam contributed to conceptualization, draft review and editing, search-result review, critical reviewer-comment analysis, and supervision.
- Patrick Mikalef contributed to conceptualization, draft and search-result/data review, critical reviewer-comment analysis, and supervision.