In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes

• AI-based models increasingly support human decision makers in domains ranging from prison sentencing and credit decisions to shopping and music recommendations.Source: §1 · p. 1
• The study asks which characteristics and information about an AI-infused decision-making process make people trust that it will produce a reasonable outcome.Source: §1 · p. 1
• A crowdsourced, scenario-based study quantitatively evaluates seven process and model factors across four facets of trust.Source: §1 · p. 1
• Interpretable models and information about how models were trained and tested were associated with increased trust ratings.Source: §1 · p. 1
• People expressed varied opinions about trustworthy AI-infused decision-making processes, motivating further study of trust factors and whether trust is always desirable.Source: §1 · p. 1
• The authors argue that trusted AI development should solicit input from builders, users, and people affected by system decisions.Source: §1 · p. 1

2.1 Definition of Trust

• Trust is defined as an attitude that an agent will help achieve an individual’s goals under uncertainty and vulnerability.Source: §2.1 · p. 2

2.2 Facets of Trust

• The paper evaluates trust in the decision-making process itself, rather than only the trustworthiness of the resulting decision.Source: §2.2 · p. 2
• Prior research operationalizes trust through varied dimensions including reliability, competence, understandability, predictability, safety, and personal attachment.Source: §2.2 · p. 2
• The study focuses on overall trustworthiness, reliability, technical competence, understandability, and personal attachment.Source: §2.2 · p. 2

• The primary question is how different aspects of an AI-infused decision-making process influence people’s feelings of trust in that process.Source: §3 · p. 2
• The study varies stakes, decider, trainer, interpretability, train-and-test-set description, social transparency, and model confidence.Source: §3 · p. 2
• Stakes compare lower-consequence decisions such as meal planning with higher-consequence decisions such as prison sentencing.Source: §3 · p. 2
• Decider distinguishes decisions made entirely by AI from decisions made by a human supported by AI guidance.Source: §3 · p. 2
• The design contrasts human-trained and automatically trained models, interpretable and black-box models, and visible versus unavailable process information.Source: §3 · p. 2

4.1 Methodology

• The study constructs 128 scenarios by crossing two levels of each of seven factors in a scenario-based design.Source: §4.1 · p. 3

4.2 Participants

• Participants were recruited through Mechanical Turk to sample viewpoints beyond the authors’ institution, with 362 English-fluent adults initially recruited.Source: §4.2 · p. 3

4.3 Procedure

• Each participant rated two randomly ordered scenarios, one lower-stakes and one higher-stakes, while other factors were randomized.Source: §4.3 · p. 3

4.4 Measures

• Trust was measured through overall trustworthiness, reliability, technical competence, understandability, and personal attachment, using adapted Likert-scale items and qualitative questions.Source: §4.4 · pp. 3–4
• Understandability was excluded from analysis because its scale had poor reliability.Source: Table 2 · p. 4, §4.4 · p. 4
• After attention screening, the final dataset contained 640 scenario ratings from 320 participants.Source: §4.4 · p. 4

5 ANALYSIS OF SCENARIO FACTORS ON TRUST FACETS

• Reliability analyses supported the trustworthiness, reliability, technical competence, and personal attachment scales, but understandability was excluded for insufficient reliability.All retained scales had α > 0.70.Source: §5 · p. 4
• ANOVA models estimated relationships between scenario factors and trustworthiness, reliability, technical competence, and personal attachment outcomes.Source: §5 · p. 4
• Qualitative open coding and clustering complemented the quantitative analysis by identifying overall themes in participants’ open-ended trustworthiness responses.Source: §5 · p. 4

5.1 Stakes

• 3.7 versus 6.3 of 7: meal planning was rated lower stakes than prison sentencing, F[1, 630] = 438.2, p < .001.Source: §5.1 · p. 4

5.2.1 Overall Ratings of Trust.

• Trustworthiness, reliability, technical competence, and personal attachment ratings all fell near the middle of their scales, suggesting possible skepticism toward AI use.Some participants expressed strongly positive or negative views.Source: §5.2.1 · p. 5

5.2.2 Stakes.

• Lower-stakes scenarios produced higher trustworthiness, technical competence, and personal attachment, while reliability did not differ by stakes.Participants generally expressed less trust in AI for higher-stakes decisions.Source: §5.2.2 · p. 5
• High-stakes scenarios elicited concerns that AI could influence important decisions despite possible technical flaws, whereas lower-stakes applications seemed less troubling.Source: §5.2.2 · p. 5

5.2.3 Model Interpretability.

• Interpretable models were rated more trustworthy, reliable, technically competent, and personally preferable than black-box models.Participants often linked interpretability to understanding why recommendations were made.Source: §5.2.3 · p. 5
• Participants described black-box decisions as difficult to trust when they could not understand the reasons or factors behind recommendations.Source: §5.2.3 · p. 5

5.2.4 Train & Test Set Description.

• Training and testing information significantly increased trustworthiness, technical competence, and personal attachment ratings.Participants frequently requested transparency about how models were developed and tested.Source: §5.2.4 · p. 5

5.2.5 Social Transparency.

• Information about decisions made for other people increased all four trust facets, although the effects were small.Participants valued comparing current recommendations with prior outcomes.Source: §5.2.5 · p. 5

5.2.6 Decider.

• Participants held mixed views about whether humans or AI should make the ultimate decision, with transparency making some participants comfortable with AI decision makers.Source: §5.2.6 · p. 6
• Some participants preferred AI because it might avoid human bias, while others distrusted it for lacking empathy, intuition, moral judgment, or compassion.Source: §5.2.6 · p. 6

5.2.7 Trainer.

• Participants generally preferred models trained by data scientists, describing self-trained systems as less trustworthy because they lacked human intervention.Source: §5.2.7 · p. 7
• When AI made the decision, human-trained models received higher trustworthiness, technical competence, and personal attachment ratings than automatically trained models.The differences were +0.2 to +0.4 points across the three facets, with very small but significant interactions.Source: §5.2.7 · p. 7

5.2.8 Model Confidence.

• When AI made the decision, providing model-confidence information raised trustworthiness, technical competence, and personal attachment ratings by +0.2 to +0.3 points.For human deciders, confidence differences were negligible, although participants’ comments strongly favored access to confidence information.Source: §5.2.8 · p. 7
• Participants often treated missing confidence information as unacceptable, though one participant considered it unimportant in a lower-stakes scenario.Source: §5.2.8 · p. 7

6 DISCUSSION

• Interpretable models and information about how AI was trained and tested produced the largest statistical effects, but trust remained complex and sometimes contradictory.Source: §6 · p. 7

6.1 ‘Black Boxes’ are Less Trustworthy

• AI was more accepted for lower-stakes decisions, yet interpretable models were the most important trust factor in both lower- and higher-stakes settings.Source: §6.1 · p. 7

6.2 In Transparent Models We Trust

• Participants preferred information about how models were created, including for higher-stakes decisions, and the authors report a significant and large trust effect.Source: §6.2 · pp. 7–8

6.3 Who Gets to Decide?

• Opinions about the ultimate decider conflicted: some accepted AI when confidence was disclosed, while others considered AI inappropriate when confidence remained secret.Source: §6.3 · p. 8
• When AI decided, participants trusted it more when people trained it; when humans decided, training method did not change trust, suggesting human involvement somewhere in the process mattered.Source: §6.3 · p. 8

6.4 Trust of Automated AI.

• Participants were skeptical of automated training methods, while views of human data scientists ranged from trusting their expertise to fearing human error or unfair intentions.Source: §6.4 · p. 8

6.5 Mitigating Human Biases

• Historical training data raised concerns about reproducing existing human and systemic biases, especially in higher-stakes decisions, although AI was also viewed as more impartial than people.Source: §6.5 · p. 8

6.6 AI’s Lack of Empathy and Morality

• AI’s lack of empathy was interpreted both as protection from emotional bias and as a limitation for higher-stakes decisions requiring moral judgment.Source: §6.6 · pp. 8–9

• The Mechanical Turk sample was limited to native English speakers from English-speaking countries, so cross-cultural validity was not assessed.Source: §7 · p. 9
• The two scenarios represented only lower- and higher-stakes consequences and may not span the full range of AI-infused decision-making contexts.Source: §7 · p. 9
• The study compared interpretable and black-box models but did not examine the many explanation methods that make black boxes more interpretable.Source: §7 · p. 9
• Future research should identify omitted trust factors and design explanations and disclosures suitable for builders, users, and people affected by AI outputs.Source: §7 · p. 9
• The high-level treatment of training and testing information leaves open which disclosures establish trust for different AI-system stakeholders.Source: §7 · p. 9

• AI’s growing role in consequential decisions makes understanding how to build trusted AI increasingly important.Source: §8 · p. 9
• The study offers quantified guidance on features associated with increased trust in AI-infused decision-making processes.Source: §8 · p. 9
• Interpretable models and information about how models were trained and tested are associated with significant improvements in trust ratings.Source: §8 · p. 10
• People’s responses were heterogeneous: some considered AI inappropriate, while others valued its perceived impartiality against human biases.Source: §8 · p. 10
• Researchers and designers should also ask when people ought to distrust AI and how systems can support appropriate mistrust.Source: §8 · p. 10