Toward quantifying trust dynamics: How people adjust their trust after moment-to-moment interaction with automation

1. INTRODUCTION

• The study asks whether trust changes differ across otherwise similar cases when automation outcomes or operators’ own performance differ.The motivating scenarios contrast trust changes after correct or incorrect system recommendations and differing final task outcomes.Source: §1 · p. 3

1.1 Trust as a Dynamic Variable

• Most prior work measured trust at experiment end, while evidence remains limited on how it strengthens or decays across moment-to-moment interactions.Prior studies report increases after automation successes and decreases after failures, with failures often exerting a larger influence.Source: §1.1 · pp. 4–5

1.2 Decision Making Heuristics/Biases and Trust Adjustment

• Outcome bias evaluates the same decision differently after a bad rather than good outcome, while contrast effects make judgments depend on prior or simultaneous stimuli.The introduction also reviews hindsight bias, but focuses the study on outcome bias and contrast effect in trust adjustment.Source: §1.2 · pp. 6–7, §1.3 · p. 8

1.3 The Present Study

• The experiment tests whether final task outcome and whether a human operator could complete it manually shape moment-to-moment trust adjustment, and hypothesizes automation failures produce larger trust decrements than successes.Source: §1.3 · p. 8
• The hypotheses predict larger trust increments after success when the operator fails, and larger decrements after failure when the operator succeeds.The study also explores how automation reliability relates to trust adjustment; reliability was set above 70% at 70%, 80%, and 90%.Source: §1.3 · p. 9

2.1 Participants

• 75 adults participated; 60 completed the experiment face-to-face and 15 remotely, with no systematic behavioral or performance differences reported between groups.Remote participants took longer, probably because of internet lags; the study received institutional review board approval.Source: §2.1 · pp. 9–10, §2 · p. 9

2.2 Apparatus and Stimuli

• Participants viewed 150 images, including 60 later targets and 90 buffer images, then completed 40 two-alternative forced-choice trials using target-distractor pairs.An interpolated memory task followed image viewing; pairs could include a distractor resembling the target or another target.Source: §2.2 · pp. 10–11
• On each trial, participants first chose the image they remembered, rated confidence, received an aid recommendation, made a final selection, and saw feedback on its correctness.The confidence scale ranged from “I’m completely guessing” to “I’m completely certain.”Source: §2.2 · pp. 12–13

2.3 Experimental Design

• The mixed design varied automation reliability between participants at 70%, 80%, or 90% and analyzed eight possible patterns of initial recognition, recommendation, and final recognition correctness.Participants did not necessarily exhibit all eight patterns.Source: §2.3 · pp. 14–15

2.4 Measures

• Trust adjustment for trial i was calculated as Trust(i) − Trust(i − 1), yielding 39 adjustments per participant across trials 2 through 40.Participants also completed a baseline trust-propensity survey and two post-experiment trust surveys.Source: §2.4 · p. 15

2.5 Experimental Procedure

• Participants completed consent and demographic and trust-propensity surveys, practiced the task, then viewed 150 new images and performed 40 trials at their assigned reliability level.They were told the aid was imperfect but not its exact reliability; post-experiment trust was also reported.Source: §2.5 · pp. 15–16

3. RESULTS

• Correct recommendations significantly increased trust in patterns 2, 3, and 7, whereas wrong recommendations significantly decreased trust in patterns 0, 4, and 5.All six one-sample tests were significant at p < 0.001.Source: §3 · p. 17, Figure 8 · p. 18, Figure 9 · p. 18
• Wrong recommendations produced larger trust-adjustment magnitudes than correct recommendations in the matched pattern 0-versus-2 and pattern 5-versus-7 comparisons.The pattern pairs differ in recommendation correctness; the reported pattern 0-versus-2 comparison was significant (t(1, 65) = 5.34, p < 0.001, Cohen’s d = 0.66).Source: §3 · pp. 18–19
• The analysis compared pattern 2 (wrong initial recognition, correct recommendation, wrong final recognition) with pattern 3 (wrong initial recognition, correct recommendation, correct final recognition), differing only in final recognition performance.Source: §3 · p. 19
• For the comparison between patterns 4 and 5, pattern 4 showed a significantly larger trust decrement (t(1, 52) = −2.63, p = .01, Cohen’s d = 0.36).Source: §3 · pp. 19–20
• When the aid recommendation was correct, participants who initially failed had a larger trust increment than those who initially succeeded (pattern 3 versus 7; p < 0.001).The comparison was t(1, 69) = 4.40, Cohen’s d = 0.53.Source: §3 · p. 20, Figure 12 · p. 21
• After a wrong recommendation and wrong final recognition, trust decrement was marginally larger for participants whose initial recognition had been correct (pattern 4 versus 0; p = .07).The comparison was t(1, 57) = 1.83, Cohen’s d = 0.24.Source: §3 · pp. 20–21
• Reliability did not significantly predict trust propensity, but reliability and propensity significantly predicted trust after trial 40 and both post-experiment trust measures.The post-experiment measures were the scales of Jian et al. (n.d.) and Muir and Moray (1996).Source: §3 · pp. 21–22

4. DISCUSSION

• Trust rises after automation successes and falls after failures, with trust decrements larger than increments, including when failures are frequent.The study observed 12 failures across 40 trials.Source: §4 · p. 22
• After an automation failure, an undesirable final recognition outcome produces a larger trust decrement; operators forgive errors to some extent when outcomes are not detrimental.Source: §4 · p. 22
• Because outcomes depend on multiple factors and automation does not determine them entirely, these findings warrant follow-up in more cognitively demanding tasks; the present task was fairly simple.Source: §4 · pp. 22–23
• The predicted larger trust increment after a correct recommendation and desirable outcome was not supported, possibly because self-serving bias led operators to under-credit the recommendation.Source: §4 · p. 23
• Automation success produces a greater trust increment when the operator fails, while failure produces a marginally greater decrement when the operator succeeds; manual ability and trust adjustment are not independent.Source: §4 · pp. 23–24
• Automation reliability did not significantly affect moment-to-moment trust adjustment, but it predicted end-of-experiment trust, plausibly through accumulated adjustments as less reliable systems produce more failures.Source: §4 · p. 24

• Understanding the trust-adjustment process enables prediction of moment-to-moment trust and can inform trust-aware adaptive automation.Source: §5 · p. 25
• Automation failures have a greater impact on trust adjustments than automation successes.Source: §5 · p. 25
• The experimental paradigm tracks trust over time while distinguishing manual ability, automation performance, and final performance.Source: §5 · p. 25
• The one-item trust scale may fail to capture all trust subdimensions.The authors suggest investigating a succinct multidimensional scale for repeated trust queries.Source: §5 · p. 25
• Performance patterns 1 and 6 were excluded from analysis because they occurred rarely.The authors suggest inducing these patterns in future research.Source: §5 · p. 25

• An automation failure leads to a larger trust decrement when the final task outcome is undesirable, and a marginally larger decrement when the operator succeeds unaided.Source: Key points · p. 26
• An automation success produces a larger trust increment when the operator fails the task independently.Source: Key points · p. 26
• The stronger effect of automation failures on trust adjustment persists when failures occur in up to 30% of trials.Source: Key points · p. 26

• One item asks whether respondents usually trust machines or automated technologies until given a reason not to.Source: Appendix A · p. 27
• The survey asks whether respondents generally distrust machines or automated technologies.Source: Appendix A · p. 27
• It asks about willingness to rely on automation for assistance and whether the respondent’s tendency to trust it is high.Source: Appendix A · p. 27
• The final item asks whether respondents can trust automation even when they know little about it.Source: Appendix A · p. 27

• Items ask whether the aid is deceptive or underhanded and whether respondents distrust its intents, actions, or outputs.Source: Appendix B · p. 28
• Other items ask whether respondents are wary of the aid or expect harmful outcomes from its actions.Source: Appendix B · p. 28
• The survey also asks about confidence in the aid, its security and integrity, and whether it is dependable and reliable.Source: Appendix B · p. 28
• A final item asks whether respondents are familiar with the automated decision aid.Source: Appendix B · p. 28

• The survey was adopted from Muir and Moray (1996) and administered after the experiment.Source: Appendix C · p. 29
• Items ask whether the aid performs its function and designed task properly, and whether participants can count on it to do its job.Source: Appendix C · p. 29
• The survey asks about moment-to-moment predictability, consistency across similar circumstances, future coping ability, and accurate responses.Source: Appendix C · p. 29
• It includes an overall trust item but omits the original survey item about trust in the aid’s display.Source: Appendix C · p. 29

• Jessie Yang is an assistant professor at the University of Michigan’s Department of Industrial and Operations Engineering and earned a 2014 PhD in Mechanical and Aerospace Engineering (Human Factors) from Nanyang Technological University.Source: Biographies · p. 36
• Christopher Schemanske is an MSE student in the University of Michigan’s Department of Industrial and Operations Engineering; he was an undergraduate there when the work was conducted.Source: Biographies · p. 36
• Christine Searle is an MS student at the University of Michigan’s Robotics Institute and earned a 2014 BA in Computer Science and Psychology from Indiana University Bloomington.Source: Biographies · p. 36