Source-linked AI summary
A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective
Tita Alissa Bach, Amna Khan, Harry Hallock, Gabriela Beltrão, Sonia Sousa
TL;DR
User trust in AI-enabled systems lacks agreed definitions, importance, and measurement. This systematic review synthesizes 23 empirical studies and finds that trust is shaped by socio-ethical, technical-design, and user factors, with user characteristics dominating.
Problem
Definitions, importance, and measurement of user trust in AI-enabled systems remain insufficiently agreed upon and studied.
Method
The paper conducts a systematic literature review to synthesize empirical evidence on user trust in AI-enabled systems.
Results
Surveys measured user trust in 59.56% of the included studies, either alone or alongside interviews or focus groups.
Takeaways & Limitations
Future research should complement subjective methods with more objective measures and examine user trust across more diverse geographical locations.
Takeaways & Limitations
Because the 23 studies used varied and sometimes specific contexts, generalizing the findings may not be feasible and requires caution.
Abstract
from arXiv · showhide
User trust in Artificial Intelligence (AI) enabled systems has been increasingly recognized and proven as a key element to fostering adoption. It has been suggested that AI-enabled systems must go beyond technical-centric approaches and towards embracing a more human centric approach, a core principle of the human-computer interaction (HCI) field. This review aims to provide an overview of the user trust definitions, influencing factors, and measurement methods from 23 empirical studies to gather insight for future technical and design strategies, research, and initiatives to calibrate the user AI relationship. The findings confirm that there is more than one way to define trust. Selecting the most appropriate trust definition to depict user trust in a specific context should be the focus instead of comparing definitions. User trust in AI-enabled systems is found to be influenced by three main themes, namely socio-ethical considerations, technical and design features, and user characteristics. User characteristics dominate the findings, reinforcing the importance of user involvement from development through to monitoring of AI enabled systems. In conclusion, user trust needs to be addressed directly in every context where AI-enabled systems are being used or discussed. In addition, calibrating the user-AI relationship requires finding the optimal balance that works for not only the user but also the system.
1. Introduction
AI-enabled systems have raised concerns about opacity, ethics, and potentially catastrophic consequences, making user trust central to trustworthy, human-centered AI. This systematic literature review examines how user trust is defined, influenced, and measured in AI-enabled systems from an HCI perspective.
- Motivation: AI applications span healthcare, banking, business, industry, and everyday life, but concerns about opaque system characteristics have increased with their uptake.The supplied passage identifies AI applications in online health platforms, banking systems, businesses, industry, and life in general.
- Motivation: Failed AI-enabled systems can produce catastrophic consequences, including discrimination and death.The passage explicitly associates failed systems with discriminatory outcomes and even death.
- Motivation: These negative consequences have reduced user trust and highlighted the importance of ethics and human accountability for AI-enabled system outcomes.The passage states that AI systems are human-designed artifacts whose creators should remain accountable, even when system logic is not understood.
- HCI perspective: Trustworthy AI requires managing risks associated with AI characteristics and moving beyond technical-centric approaches toward human-centered design, a core HCI principle.The passage directly connects trustworthiness, risk management, human-centered approaches, and HCI.
- Research focus: Fostering and maintaining user trust is presented as key to calibrating the user-AI relationship, achieving trustworthy AI, and unlocking AI’s societal potential.The passage also notes that trust, trustworthy, and trustworthiness may lack a clear focus in user-AI interactions, while trust is a social-technical construct rather than a single construct.
- Research focus: The review addresses how user trust in AI-enabled systems is defined, which factors influence it, and how it can be measured.These aims correspond to the review’s three research questions and its stated contribution to HCI literature.
2. Methodology
The study used a systematic literature review aligned with PRISMA standards to rigorously synthesize empirical evidence on user trust in AI-enabled systems. Searches across two digital libraries applied defined keywords and eligibility criteria, yielding 23 included articles from 493 records.
- Review design: A systematic literature review was chosen to summarize empirical evidence rigorously, systematically, and with reduced bias while supporting study replicability.The method was selected after an initial search revealed varied approaches to studying user trust in AI-enabled systems.
- Search strategy: The review followed PRISMA standards and searched EBSCO Discovery Service and Web of Science using research-question-driven terms refined through pilot searching and ACM CSS 2012.The search strings were designed to include relevant articles comprehensively.
- Eligibility and selection: 493 articles were identified, with inclusion restricted to English empirical scientific articles or conference proceedings published from January 1, 2011 to May 15, 2021.Eligible studies also required a clear methodology, factors influencing user trust in AI-enabled systems, and an available full-text version.
- Analysis and synthesis: The final sample comprised 23 articles, whose data were independently extracted, synthesized through author meetings, and quality checked for consistency.Extracted information included study context, AI-enabled system, trust definition, methodology, trust measurement, participants or dataset, and influencing factors.
3. Results
The review included 23 empirical articles spanning diverse AI-enabled systems, with studies concentrated after 2018, in the USA and Germany, and in Robotics and E-commerce. General AI/ML and automated algorithms were the most common system types, while most articles assessed, predicted, or augmented antecedents, predictors, or critical dimensions.
- Publication and study coverage: 52.17% of the 23 included articles were published after 2018.This distribution was reported in Table 3.
- Publication and study coverage: 56.52% of studies were conducted in the USA and Germany, while 52.17% focused on Robotics and E-commerce.The included articles examined various types of AI-enabled systems.
- AI-enabled system types: 30.43% of the included articles examined general AI/ML and automated algorithms, the most common AI-enabled system types.The review covered diverse AI-enabled systems beyond these most common types.
- Research focus: Almost 78.36% of articles focused on assessing, predicting, or augmenting antecedents, predictors, or critical dimensions.The supplied passage ends after “critical dimen,” so the final term is preserved only to the extent provided.
3.1. RQ1: How is user trust in AI-enabled systems defined?
Trust in AI-enabled systems was defined inconsistently across the reviewed articles: some adopted established definitions, while others only conceptualized trust or did neither. Mayer’s definition was the most frequently used among the explicitly reported frameworks, alongside Lee and See’s definition and one author-developed definition.
- Definition coverage: Seven articles provided trust definitions, while eight conceptualized trust without defining it and eight did neither.Thus, fewer than half of the reviewed articles explicitly defined trust.
- Adopted definitions: Four articles used Mayer’s trust definition.The cited studies were Foehr and Germelmann, Glikson and Woolley, Lin, and Thielsch et al.
- Adopted definitions: Two articles used Lee and See’s trust definition, while one developed its own definition using trustworthy characteristics from Avizienis et al.The Lee and See definition was used by Söllner and Zhou et al.; Yan et al. developed the combined definition.
3.2. RQ2: What factors influence user trust in AI-enabled systems?
User trust in AI-enabled systems was influenced by socio-ethical considerations, technical and design features, and user characteristics. User characteristics dominated the findings, while trust also depended on system transparency, reliability, user readiness, prior experience, and alignment with user expectations.
- Main themes: Three themes influenced trust: socio-ethical considerations, technical and design features, and user characteristics, identified by 8, 12, and 22 articles, respectively.User characteristics were the most frequently identified influence across the 23 included articles.
- Socio-ethical considerations: Trust was supported by implementation readiness, open communication, ongoing feedback, and clearly assigned accountability for potential harm.Suggested mechanisms included preparing the implementation environment, maintaining communication, and establishing ethical-legal boundaries.
- Technical and design features: Virtual-agent trust increased with human-like benevolence, immediacy, social presence, psychological closeness, integrity, and supportive text or speech outputs.Examples included smiling, showing interest, repeatedly satisfactory task fulfillment, additional text or speech, and text rather than synthetic voice.
- Technical and design features: For AI/ML systems, trust was influenced by explanations, contextual and performance information, interactive risk tools, prediction correctness, and system integrity.Relevant information included algorithm operation, AI actions, reliability, model performance, feature influence, risk factors, and risk trends.
- User characteristics: User characteristics dominated findings across inherent characteristics, acquired characteristics, attitudes, and external variables, while personality, gender, self-trust, and education shaped trust.Low Openness was associated with the highest trust, women were more likely to report higher trust, and familiarity increased trust over time.
- User characteristics and system context: Trust depended on acceptance, readiness, prior provider experiences, system reliability, information credibility, usefulness, and matching user expectations.Early involvement, training, empowerment, uncertainty reduction, and addressing mismatches between expectations and experiences were suggested to foster trust and prevent mistrust.
3.3. RQ3: How is user trust in AI-enabled systems measured?
User trust in AI-enabled systems was measured primarily through surveys, often using researcher-developed questionnaires, while qualitative methods were the second most common approach. Participant samples varied widely across the reviewed studies.
- Measurement methods: Surveys were used in 16 studies (69.56%), either alone or combined with interviews or focus groups, making them the predominant trust-measurement method.Of these studies, 12 (75%) developed their own questionnaires, two (12.5%) combined original and previously developed questionnaires, and two (12.5%) used previously developed questionnaires.
- Measurement methods: Qualitative methods, including interviews and focus groups, measured user trust in six studies (26.09%), either alone or combined with another method.They were the second most common measurement approach.
- Participants: Participant samples in 20 articles ranged from 21 to 3423 participants (M = 326.80).Fourteen articles reported 23.08–61.86% female participants (M = 45.62%), while two reported participants identifying as other than male or female.
4. Discussion
The review finds that trust has multiple context-dependent definitions, while user characteristics, interactions, system features, and socio-ethical conditions shape trust and its measurement. It recommends context-sensitive definitions, continuous user involvement, and cautious interpretation because included studies and methods vary.
- Trust definitions: Only seven of 23 studies explicitly define trust; eight conceptualize it, while nine provide neither.Trust is difficult to define or generalize because it is abstract, dynamic, and context-dependent.
- Trust definitions: The review recommends selecting the trust definition most appropriate to the context rather than comparing definitions.The appropriate definition may depend on factors such as the risk an AI output poses to users.
- Influences on trust: User characteristics dominate the findings, supporting continuous user involvement from AI system development through implementation and monitoring.User trust may increase over time through more user-system interactions, meaning low initial trust can improve.
- Influences on trust: Trust-influencing factors vary with user and system contexts, so system features should be tailored to targeted users.Socio-ethical conditions, including explanations of AI systems, can support trusted user-AI relationships.
- Trust measurement: 69.56% of included studies developed and used their own questionnaires, making surveys the most common trust-measurement method.Qualitative methods were the second most used and are considered appropriate for exploring complex topics such as trust.
- Limitations: Findings should be generalized cautiously because the 23 studies use varied and sometimes highly specific contexts, definitions, and search coverage.The review notes that additional trust definitions and relevant studies may have been excluded.
5. Conclusions and future aims
Future research should further examine how trust concepts and influencing factors develop over time and shape evolving user-AI relationships. It should also complement perception-based methods with more objective measures of user trust.
- Future research priorities: Future studies should investigate trust concepts and influencing factors at different points in time as user-AI interactions evolve into relationships.The review identifies time as a factor requiring more detailed investigation.
- Future research priorities: Surveys, interviews, and focus groups are common trust-measurement methods but depend on relatively subjective user perceptions.These methods may not provide fully objective insight into user trust.
- Future research priorities: Future research may add psychophysiological signals to quantitative or qualitative methods for more objective insight into user trust.Psychophysiological signals are proposed as a possible complementary measurement approach.
ORCID · About the authors
The authors bring complementary expertise in human factors, trustworthy and transparent systems, cognitive neuroscience, human-centered trust, healthcare AI, data governance, and interaction design. Their backgrounds span research, engineering, product development, project management, and funded HCI initiatives.
- About the authors: Tita Alissa Bach holds a doctorate in behavioural and social sciences from Groningen University, focusing on Human Factors and Organisational Psychology.Her research applies Human Factors principles to technology-use environments and organizations.
- About the authors: The author team combines expertise in human factors, adaptive systems, trustworthy AI, health-data governance, cross-cultural trust, and interaction design.These areas are represented across the authors’ stated research and professional activities.
- About the authors: Amna Khan is a Junior Researcher and PhD student in Information Society Technologies at Tallinn University.She researches adaptive systems for knowledge reuse and trustworthy, transparent systems, and holds a 2017 master’s degree in Mechatronics Engineering.
- About the authors: Harry Hallock holds a doctorate in Cognitive Neurosciences from the University of Sydney and has experience in product development and large-scale project management.His current work addresses trustworthy AI in healthcare, federated health-data networks, and health-data governance, management, and quality assurance.
- About the authors: Gabriela Beltr~ao is an Information Society Technologies candidate at Tallinn University whose research examines trust in technology from a human-centered perspective.She focuses on cross-cultural differences in trust and their implications for design.
- About the authors: Sonia Sousa is an Associate Professor of Interaction Design at Tallinn University and leads the Joint Online MSc in Interaction Design.Her funded projects include NGI-Trust, CHIST-ERA, Horizon 2020, and AFOSR.