Source-linked AI summary
Putting ChatGPT's Medical Advice to the (Turing) Test
Oded Nov, Nina Singh, Devin Mann
TL;DR
The study addresses the need for empirical investigation of patient communication with AI-based chatbots. It examines chatbot use across health-question scenarios and finds responses weakly distinguishable from provider responses, with laypeople trusting chatbots more for lower-risk questions.
Problem
Patient communication with human providers requires empirical investigation when AI-based chatbots are used.
Source: Discussion · p. 6Method
The study considers chatbot support for patient questions about treatment plans, medication titrations, potential side effects, and higher-risk clinical interactions.
Source: Discussion · pp. 7–8Results
ChatGPT responses to patient questions were weakly distinguishable from provider responses, with identification rates of 65.5% for chatbot responses versus 65.1% for provider responses.
Source: Discussion · p. 7, Conclusion · p. 9Takeaways & Limitations
Laypeople trusted chatbots to answer lower-risk health questions, while identifying appropriate deployment scenarios remains an important next step.
Source: Discussion · p. 7, Conclusion · p. 9ChatGPT was not trained on medical data and could be inferior to medically trained systems.
Source: Discussion · p. 8
Abstract
from arXiv · showhide
Objective: Assess the feasibility of using ChatGPT or a similar AI-based chatbot for patient-provider communication. Participants: A US representative sample of 430 study participants aged 18 and above. 53.2% of respondents analyzed were women; their average age was 47.1. Exposure: Ten representative non-administrative patient-provider interactions were extracted from the EHR. Patients' questions were placed in ChatGPT with a request for the chatbot to respond using approximately the same word count as the human provider's response. In the survey, each patient's question was followed by a provider- or ChatGPT-generated response. Participants were informed that five responses were provider-generated and five were chatbot-generated. Participants were asked, and incentivized financially, to correctly identify the response source. Participants were also asked about their trust in chatbots' functions in patient-provider communication, using a Likert scale of 1-5. Results: The correct classification of responses ranged between 49.0% to 85.7% for different questions. On average, chatbot responses were correctly identified 65.5% of the time, and provider responses were correctly distinguished 65.1% of the time. On average, responses toward patients' trust in chatbots' functions were weakly positive (mean Likert score: 3.4), with lower trust as the health-related complexity of the task in questions increased. Conclusions: ChatGPT responses to patient questions were weakly distinguishable from provider responses. Laypeople appear to trust the use of chatbots to answer lower risk health questions.
Background
The study examines whether ChatGPT can feasibly answer patient portal messages as providers face rising messaging demands and limited dedicated time. It measures public ability to distinguish chatbot from provider responses and trust in chatbot functions.
Background
- Increasing patient-message burdens, EHR work, provider burnout, and limited reimbursement motivate exploring chatbot support for patient-provider communication.Source: Background · p. 4
- The study assesses whether people can distinguish AI-generated from provider-generated responses to patients’ health questions.Source: Background · p. 4
- The study also characterizes public trust in chatbot functions and discusses implications for adopting AI-based chatbots in patient messaging portals.Source: Background · p. 4
Methods
Researchers compared ChatGPT and provider responses to ten representative patient-provider interactions, then surveyed adults on source identification and chatbot trust.
Methods
- Ten representative non-administrative patient-provider interactions were extracted from the EHR for comparison.Source: Methods · p. 5
- Patients’ questions were entered into ChatGPT, which was prompted to produce responses approximately matching provider word counts after identifying details were removed and typos corrected.Source: Methods · p. 5
- A US representative sample of 430 adults evaluated whether each response came from a provider or ChatGPT, with five responses from each source and financial incentives for correct classification.Source: Methods · p. 5
- Participants also rated trust in chatbots’ use for patient-provider communication on a 1–5 Likert scale.Source: Methods · p. 5
Results
Participants distinguished chatbot and provider responses only moderately well, while trust in chatbots was positive overall but declined as task complexity increased.
Results
- Response classification varied by question, ranging from 49.0% to 85.7% correct.Source: Results · p. 6
- 65.5% of chatbot responses and 65.1% of provider responses were identified correctly on average.Source: Results · p. 6
- Trust in chatbots was positive on average but decreased as the health-related complexity of the task increased.Source: Results · p. 6
- Trust was not significantly correlated with participant demographics or ability to identify chatbot versus human responses.Source: Results · p. 6
Discussion
ChatGPT responses were weakly distinguishable from human-provider responses, suggesting potential use in selected patient-communication scenarios. Trust was higher for lower-complexity or administrative tasks, while clinical complexity, model choice, disclosure, and limited study scope constrain deployment.
Study findings
- 65.5% versus 65.1% distinguishability rates indicate that chatbot and human responses were difficult for laypeople to distinguish.The study used 50% as the random-distinguishability benchmark and 100% as perfect distinguishability.Source: Discussion · p. 7
Trust and acceptance
- Trust decreased as medical complexity increased, with highest acceptance for scheduling and lowest acceptance for treatment advice.The paper notes that this pattern is broadly consistent with prior studies.Source: Discussion · p. 7
Potential use cases
- Chatbots may support common chronic-disease management, including questions about treatment plans, medication titration, and side effects.The paper describes generalized chatbots as an opportunity beyond specialized scenarios such as symptom triage and post-chemotherapy education.Source: Discussion · p. 7
Operational implications
- Chatbots could interact directly with patients or assist clinicians by drafting and transforming documentation, shifting provider work toward curation rather than creation.These models are presented as possible ways to support communication, access to care, and personalized education.Source: Discussion · pp. 6, 8
Deployment models
- Clinical complexity should shape deployment: higher-risk interactions may require clinician editing, whereas lower-risk situations may permit direct patient-chatbot interaction.A classification workflow could answer administrative questions directly while drafting health-question responses for provider approval.Source: Discussion · p. 8
Limitations and next steps
- Interpretation is limited because ChatGPT was not medically trained, lacked specialized prompting, and was evaluated using ten questions from one provider.The authors call for studies using larger numbers of real-world questions and responses.Source: Discussion · p. 8
Conclusion
ChatGPT responses were weakly distinguishable from provider responses, and laypeople trusted chatbots more for lower-risk health questions. The authors call for continued study as chatbots expand from administrative to clinical healthcare roles.
- ChatGPT responses to patient questions were weakly distinguishable from provider responses.Source: Conclusion · p. 9
- Laypeople trusted chatbots to answer lower-risk health questions.Source: Conclusion · p. 9
- The authors recommend continued study of patients’ objective and emotional interactions with chatbots as they move into clinical roles.Source: Conclusion · p. 9
Funding
The authors received financial support from the US National Science Foundation, which had no further role in the submitted work.
- The authors received financial support from the US National Science Foundation for the submitted work.Source: Funding · p. 9
- The funding source had no further role in the submitted work.Source: Funding · p. 9
- The reported NSF awards were 1928614 and 2129076.Source: Funding · p. 9
Figures and Tables
The paper presents a figure on correct-response distributions and a table on health-chatbot trust questions and responses. Participant response identification was scored from 0–10.
- Figure 1 presents the distribution of correct responses.Source: Figure 1 · p. 12
- Participant identification performance was scored from 0–10 based on the number of responses identified correctly.Source: Figures and Tables · p. 12
- Table 1 presents health chatbot trust questions and responses.Source: Table 1 · p. 12