Source-linked AI summary
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Michael Fitzke, Mark Parkinson, Diane Wilson, Paul Fisher, Eric Horvitz, Kori Inkpen, Besmira Nushi
TL;DR
The paper asks how the timing of AI advice in clinical imaging affects human decision making, an important interaction-design question with limited prior evidence. It compares immediate AI display with advice shown after a provisional assessment in a study of 19 veterinary radiologists. Immediate display was associated with stronger alignment, including when AI advice was erroneous, while task time was comparable across workflows.
Problem
Evidence is limited on how making AI inferences available before versus after a human’s diagnostic review affects decision making in clinical imaging.
Source: §1 · p. 2Method
The authors compared two AI-display workflows in a study where 19 veterinary radiologists assessed X-ray findings with an AI tool.
Source: §1 · p. 2Results
Radiologists’ diagnoses aligned more with AI when advice appeared immediately, with marginal diagnostic-performance gains and stronger alignment even for inaccurate AI advice; time spent was comparable across workflows.
Source: §1 · p. 2, §6 · p. 15Takeaways & Limitations
The findings suggest that workflow design matters: immediate integration may fit radiologists’ decision processes more smoothly, but can also increase inappropriate reliance on erroneous advice.
Source: §6 · p. 15The experiment provided only one X-ray image per patient and no clinical background information, which may have led radiologists to behave differently than they would with typical clinical information and views.
Source: §5 · p. 14
Abstract
from arXiv · showhide
Details of the designs and mechanisms in support of human-AI collaboration must be considered in the real-world fielding of AI technologies. A critical aspect of interaction design for AI-assisted human decision making are policies about the display and sequencing of AI inferences within larger decision-making workflows. We have a poor understanding of the influences of making AI inferences available before versus after human review of a diagnostic task at hand. We explore the effects of providing AI assistance at the start of a diagnostic session in radiology versus after the radiologist has made a provisional decision. We conducted a user study where 19 veterinary radiologists identified radiographic findings present in patients' X-ray images, with the aid of an AI tool. We employed two workflow configurations to analyze (i) anchoring effects, (ii) human-AI team diagnostic performance and agreement, (iii) time spent and confidence in decision making, and (iv) perceived usefulness of the AI. We found that participants who are asked to register provisional responses in advance of reviewing AI inferences are less likely to agree with the AI regardless of whether the advice is accurate and, in instances of disagreement with the AI, are less likely to seek the second opinion of a colleague. These participants also reported the AI advice to be less useful. Surprisingly, requiring provisional decisions on cases in advance of the display of AI inferences did not lengthen the time participants spent on the task. The study provides generalizable and actionable insights for the deployment of clinical AI tools in human-in-the-loop systems and introduces a methodology for studying alternative designs for human-AI collaboration. We make our experimental platform available as open source to facilitate future research on the influence of alternate designs on human-AI workflows.
1 INTRODUCTION
This study examines how the timing of AI advice shapes veterinary radiologists’ decisions and use of AI. It finds that immediate AI display increases alignment, but can also increase anchoring on erroneous advice; the authors report comparable task time across workflows.
1 INTRODUCTION
- The study compares AI shown alongside X-rays with AI revealed only after radiologists record provisional diagnoses, examining anchoring, performance, agreement, time, confidence, and perceived usefulness.The study involved 19 veterinary radiologists and an AI tool applied to real-world X-ray cases.Source: §1 · p. 2
- Radiologists’ diagnoses aligned more with AI when advice appeared immediately, with marginal diagnostic-performance gains but greater risk of inappropriate reliance on erroneous advice.Alignment was similar across workflows for findings considered critical for the animal.Source: §1 · p. 2
- The one-step workflow produced stronger AI alignment, while two-step participants rarely revised provisional diagnoses after seeing differing AI advice; time was comparable and one-step participants rated AI more useful.Participants in the one-step workflow also reported greater confidence and speed gains from AI assistance.Source: §1 · p. 2
- The study contributes an analysis of alternative workflows, deployment takeaways, and an open-source experimental platform for future human-in-the-loop clinical-imaging research.The platform is available at http://aka.ms/Exp-HAIC.Source: §1 · p. 2
2 BACKGROUND
Prior human-AI team research has examined AI explanations, communicated accuracy, and model updates, while clinical-imaging studies have emphasized team performance. This paper focuses on workflow timing with domain experts and considers both anchoring and the cognitive costs of revisiting decisions.
2.1 Analyses of Human-AI Teams
- Human-AI collaboration research has studied explanations, communicated accuracy, and model updates; this study adds workflow configuration using domain experts on a medical diagnostic task.The authors distinguish their expert participants from lay participants in prior studies.Source: §2.1 · p. 3
- Clinical-imaging studies have mostly compared human-only and human-AI diagnostic performance, while some examine assistance types and report both benefits and risks.Reported risks include extra time without accuracy gains, reduced performance from erroneous advice, and ignored AI recommendations.Source: §2.1 · p. 3
2.2 Workflow Considerations for Human-AI Teams
- The authors hypothesized that AI shown during initial analysis would exert stronger anchoring influence than AI shown after a provisional assessment.The hypothesis draws on prior evidence that people may weight earlier assessments more strongly than later evidence.Source: §2.2 · p. 3
- Prior workflow-ordering studies report mixed findings: provisional predictions sometimes improve performance, while another study found no workflow differences; this study tests the question with professional radiologists.The authors note experts may have established decision processes and may be reluctant to use AI advice.Source: §2.2 · p. 4
- A two-step workflow requires reopening a completed analysis, which the authors frame as adding cognitive effort analogous to task switching, interruption, and recovery.Prior cognitive-forcing approaches can reduce AI reliance but have required additional decision time.Source: §2.2 · p. 4
3 METHODS
The study used a web-based experiment to compare AI shown with X-rays against AI revealed after provisional diagnoses. Veterinary radiologists assessed 40 dog X-rays, and the analysis combined diagnostic, behavioral, timing, and survey measures.
3 METHODS
- The experiment used past patient X-rays and received institutional review board approval from Mars.The authors describe the study setup, procedures, measures, analyses, and platform in this section.Source: §3 · p. 4
3.1 Experimental Task
- Radiologists assessed 33 specified findings across 40 dog X-rays, estimating likelihood, making present-or-absent judgments, and indicating whether they would seek a colleague’s second opinion.Images came from a benchmark annotated independently by 10–13 expert radiologists; 40 were selected to oversample cases with low agreement, and about 7% of findings were ground-truth positive.Source: §3.1 · pp. 4–5
3.2 Experimental Design
- In the one-step workflow, X-rays and AI inferences appeared together; in the two-step workflow, radiologists recorded provisional diagnoses before AI was revealed and could then revise them.The 19 completing participants reviewed the same two series of 20 images; workflow assignment balanced experience and prior exposure to AI data labeling.Source: §3.2 · pp. 5–6
3.3 Measures and Statistical Analysis
- The interface displayed the X-ray, finding names, AI confidence, and controls for likelihood, diagnosis, and second-opinion responses; AI flags were hidden during the two-step initial review.The study measured AI alignment, diagnostic performance, inter-rater reliability, time, confidence proxies, workload, usefulness, and future use, using bootstrap summaries, permutation tests, and regression analyses.Source: Figure 1 · p. 6, §3.2 · p. 6, §3.3 · p. 7
3.4 Experimental Platform
- The Django platform logs image-level decisions, elapsed time, and subjective responses, and supports researcher-supplied image data, diagnoses, and AI-flag thresholds.It also supports comprehension checks and post-task surveys.Source: §3.4 · p. 8
4 RESULTS
The one-step workflow produced stronger alignment with AI flags and higher inter-rater reliability, but only marginal diagnostic-performance gains. Time on task did not differ, while two-step participants rated AI advice less useful.
4.1 Alignment Between Participants and AI
- AI alignment was 91% in the one-step and 89% in the two-step workflow, with significantly stronger alignment in one-step; flag-related anchoring was stronger, mainly for non-critical findings.The discontinuity estimates at the 0.6 flag threshold were 0.24 [0.03] and 0.11 [0.03] for one- and two-step workflows, respectively.Source: §4.1 · pp. 8–9
4.2 Diagnostic Performance
- The AI outperformed participants on most metrics, while one-step workflow gains were marginal and participants performed worse when AI flags were wrong.Positive predicted values were 42% versus 39% in one- and two-step workflows, a difference that was not statistically significant.Source: §4.2 · pp. 10–11, Table 1 · p. 10
4.3 Inter-rater Reliability
- Inter-rater reliability was higher in the one-step workflow overall, with Fleiss’ kappa of 0.55 versus 0.49, and the largest gap occurred for non-critical findings.For findings with expected low disagreement, kappas were 0.37 and 0.35 in the one- and two-step workflows.Source: §4.3 · p. 11
4.4 Time Spent on Decision Making and Confidence
- Time on task did not differ between workflows, and subjective likelihood estimates were comparable despite workflow differences in second-opinion patterns.Median times were 152 and 139 seconds for two-step and one-step workflows; average likelihood estimates for AI-flagged findings identified by participants were 0.89 and 0.90.Source: §4.4 · pp. 11–13
4.5 Perceptions about AI inferences
- Two-step participants rated AI utility lower across accuracy, confidence, and speed, and were less willing to use the tool, though the future-use difference was not significant.AI usefulness was reported for 36% of images in one-step and 17% in two-step; participants most often found it helpful for minor or incidental findings.Source: §4.5 · pp. 13–14
5 DISCUSSION
The discussion identifies limits to the study’s clinical realism and calls for further analysis of interaction design. It also considers modifications that could balance satisfaction with AI and the risk of anchoring on erroneous advice.
Task realism
- The study omitted patient history and additional X-ray views, and participants knew their decisions would not affect treatment, so behavior may differ in practice.Onboarding was also brief; real-world deployment should ideally include longer, more detailed instruction about the tool and its intended use [47] [77].Source: §5 · p. 14
Choices of interaction design
- Alternative interfaces, selective review, and confidence-based case ordering warrant study because interaction-design choices may affect results.Agreement between participants and AI flags increased with AI confidence; the authors suggest radiologists might first review findings with the highest AI uncertainty.Source: §5 · p. 14
- The AI-flag threshold, set at 0.6 in this study, can influence final diagnostic decisions, while learning effects may affect trust, ease of use, and timing analyses.The platform was used over two sessions; learning effects might have been more visible with more sessions.Source: §5 · p. 15
Appropriate reliance
- The authors propose written rationales or a second radiologist for disagreements on critical findings when AI confidence is far from the decision boundary.They caution that such checks must be balanced against prior findings that cognitive forcing functions decrease trust and willingness to work with AI assistants [10].Source: §5 · p. 15
6 CONCLUSIONS
Radiologists’ diagnoses aligned more with AI advice when it appeared immediately, with marginal diagnostic-performance gains and higher inter-rater reliability in that workflow. The authors emphasize interaction design while noting that more remains to be explored.
6 CONCLUSIONS
- Immediate AI display produced greater diagnostic alignment, marginal performance gains, and higher inter-rater reliability than displaying AI after provisional assessment.Alignment was similar across workflows for findings considered critical for the animal.Source: §6 · p. 15
- Stronger alignment in the one-step workflow also occurred with inaccurate AI inferences, suggesting greater inappropriate reliance and anchoring.The authors suggest the one-step workflow fits radiologists’ current decision-making processes more smoothly, while proposing that task switching and interruption may help explain dissatisfaction with the two-step workflow.Source: §6 · p. 15
- The study examines whether AI inferences are available immediately or only after an AI-free analysis, and the authors state that much remains to be explored.Source: §6 · p. 15