PaperlayerView exact revision on arXiv

Source-linked AI summary

Why and How People Check Generative AI Output for Mistakes

Patrick Gage Kelley, Derrick Feldmann, Reena Jana, Colleen Thompson-Kuhn, Allison Woodruff

arXiv:2609.38186v1cs.HC

TL;DR

GenAI can make mistakes, but evidence about what the public knows and how people check results has been limited. A representative US survey of 1,503 respondents examined these issues and found high awareness of mistakes, alongside reported checking of outputs.

  • Problem

    Evidence about public understanding of GenAI mistakes and how everyday users check results has been limited.

    Source: Related Work · p. 2

  • Method

    The authors surveyed a representative sample of 1,503 people in the United States about GenAI mistakes, attitudes, and checking behavior.

    Source: Introduction · p. 1

  • Results

    70% of respondents said GenAI makes mistakes at least sometimes, and respondents reported checking its output at least some of the time.

    Source: Introduction · p. 1

  • Takeaways & Limitations

    The authors identify opportunities for messaging that educates users about mistakes and supports human-AI verification, while noting users’ interest in providing feedback.

    Source: Future Work · p. 10

    The survey was conducted only in the US, and its findings reflect a moment in time as public opinion may shift with the technology.

    Source: Limitations · p. 3

Abstract

from arXiv · show

Generative AI output can contain errors, such as hallucinations, non-responsive results, or otherwise inaccurate or potentially harmful content. To explore the public's emerging understanding, attitudes, and behavior regarding such mistakes, we ran an online survey in the United States with 1,503 respondents, with a representative sample of the population. We report high public awareness of generative AI mistakes. Further, many respondents report checking generative AI output, for example, by comparing results with other online resources. We conclude with guidance for explanations and in-product disclosures about generative AI mistakes.

Introduction

The paper examines what the public knows about GenAI mistakes and how people respond to them. A representative US survey finds widespread awareness, reported checking of outputs, and more favorable responses to guidance than to general warnings.

Introduction

  • The paper addresses uncertainty about what the public knows of GenAI mistakes and what people do to detect or mitigate them.The paper identifies hallucinations, non-responsive results, and inaccurate or potentially harmful content as examples of mistakes.Source: Introduction · p. 1
  • 70% of respondents said GenAI makes mistakes at least sometimes.Respondents also recognized several kinds of mistakes, including wrong facts, fabricated information, and failure to follow prompts.Source: Introduction · p. 1
  • The study examines public understanding, attitudes, and behavior regarding GenAI mistakes through an online survey of 1,503 US respondents in a representative sample.The survey was fielded in May 2025.Source: Introduction · p. 1
  • Respondents reported checking GenAI output at least some of the time, for example by comparing results with other online resources.The paper also reports antipathy toward general warnings about mistakes, with more favorable responses to guidance to check results and provide feedback.Source: Introduction · p. 1

Related Work

Prior research documents public attitudes toward AI and concerns about mistakes, but evidence about how everyday users check GenAI results remains limited. This study situates its contribution against work on trust, error awareness, and verification practices.

Related Work

  • Related work describes rapid GenAI adoption and substantial personal and professional use, while noting research on error-related attitudes and behavior remains sparse.The paper also reviews wider concerns about AI and reports of perceived benefits from workplace use.Source: Related Work · p. 1
  • Prior work describes mixed public attitudes toward AI, concerns about its effects, and low or moderate awareness of visible GenAI mistakes.Reported concerns include misinformation; studies also document people questioning AI accuracy and recognizing hallucinations or low-quality output.Source: Related Work · p. 2
  • Research on AI trust includes findings of low/high trust in GenAI and distrust of AI summaries, alongside studies of over-reliance and potential errors.The paper frames awareness of potential errors as important in this related work.Source: Related Work · p. 2
  • Prior verification studies cover knowledge workers, data analysts, and blind spreadsheet users, including source-checking and search-engine use.The cited work also reports barriers such as limited time and differences in blind users’ practices compared with sighted users.Source: Related Work · p. 2
  • Proposed verification supports include AI literacy and interfaces using warnings, chain-of-thought reasoning, apologies, or warn-verify-audit designs.Prior studies report improvements on measures such as error detection, speed, and favorability.Source: Related Work · p. 2
  • Research on public perception of GenAI mistakes and everyday users’ checking practices remains limited.The paper describes its study as complementing prior work by investigating verification attitudes and behaviors in a general population.Source: Related Work · p. 2

Methodology

The study uses a US online survey to investigate beliefs about GenAI mistakes, checking behavior, and reactions to explanatory messaging. It combines closed-form questions, open-ended responses, and a highlighter exercise, with quantitative and qualitative analysis.

Methodology

  • The survey was designed to address four questions about perceived GenAI mistakes, their kinds and frequency, checking behavior, and useful explanations or in-product statements.The questions also ask why and how people check GenAI results.Source: Methodology · p. 2

The Questionnaire

  • Respondents received a definition of GenAI and answered questions on demographics, attitudes, familiarity, use, mistakes, and explanatory messaging.The highlighter exercise let respondents mark words in hypothetical statements as liked/helpful or disliked/not helpful; questions were closed-form, open-ended, or part of that exercise.Source: The Questionnaire · pp. 2–3

Survey Deployment

  • The online survey included 1,503 US respondents and was fielded in English and Spanish from May 8–14, 2025.The sample was representative of US Census data on age, gender, race/ethnicity, region, and household income; 163 respondents unfamiliar with GenAI skipped some questions, including those on AI use.Source: Survey Deployment · p. 3

Data Processing and Analysis

  • The researchers used inductive analysis for emerging themes and modeled closed-form responses with multivariable logistic regression reported as adjusted odds ratios.Open-ended responses were reviewed for themes; the last author developed and applied a codeframe to responses about checking, in consultation with the first author.Source: Data Processing and Analysis · p. 3
  • Data processing included quality checks, removal of low-quality or incomplete responses, and translation of Spanish responses into English for analysis.The paper reports that 338 respondents were removed before the survey, 57 after quality checks, and 316 who dropped out were removed from the final dataset.Source: Data Processing and Analysis · p. 3

Limitations

The authors identify methodological risks, limited geographic scope, and the time-sensitive nature of their findings as limitations. Responses reflect a particular moment as GenAI and public understanding continue to change.

Limitations

  • Survey responses may be affected by translation problems, misunderstanding, satisficing, or plagiarized open-ended answers.The researchers say they sought to minimize these risks through open- and closed-form questions and data quality checks.Source: Limitations · p. 3
  • Because the survey was conducted only in the US, its findings may not represent other geographies.Source: Limitations · p. 3
  • The findings capture a moment in a rapidly changing GenAI landscape, and public opinion may shift as the technology evolves.Source: Limitations · p. 3

Findings

Respondents broadly recognized GenAI mistakes, and many said they check results, often against other sources. They favored guidance to check results over general warnings that GenAI makes mistakes.

Findings

  • Only 11% found GenAI results very trustworthy, while 89% reported limited trust, distrust, or uncertainty.Excitement and perceived usefulness were associated with higher trust; concern was associated with greater distrust.Source: Findings · p. 5

Occurrence of GenAI Mistakes

Types and Frequency of GenAI Mistakes

Checking GenAI Mistakes

  • 75% of respondents who answered reported checking GenAI results at least sometimes, while 24% checked rarely or not at all.28% broadly said yes, and 13% said they always check.Source: Checking GenAI Mistakes · p. 6
  • Reasons for not checking included low-stakes use, trust in GenAI, and the time cost of checking.Some respondents said their trust had grown through experience, while others said checking could defeat GenAI's efficiency purpose.Source: Checking GenAI Mistakes · p. 7
  • Respondents checked results for accuracy by cross-referencing other sources (25%), and also reviewed relevance, grammar, tone, and appropriateness.Some checked when results seemed suspicious, concerned them, or might cause embarrassment if shared incorrectly.Source: Checking GenAI Mistakes · pp. 7–8

Explanations and In-Product Statements

  • Respondents favored explanations highlighting improvement, user feedback, and checking results; they disliked general mistake warnings, especially claims of inconsistent results.Many also wanted further information about verification, accuracy, training inputs, and societal impacts.Source: Explanations and In-Product Statements · p. 8

Discussion and Conclusions

The paper offers disclosure guidance informed by respondents’ reactions and identifies questions for future research. It recommends constructive, actionable messages while noting that users’ ability to catch difficult errors is limited.

Discussion and Conclusions

  • The authors frame the five insights as guidance for developers, regulators, and civil society considering disclosure practices.The guidance aims to help users grasp GenAI limitations and consider possible actions.Source: Guidance for Disclosures of GenAI… · p. 8

Guidance for Disclosures of GenAI Mistakes

  • Respondents disfavored repeated or vague warnings about mistakes, although new users may need reminders more often than regular or heavy users.The authors recommend balancing necessary acknowledgment of errors with constructive framing.Source: Guidance for Disclosures of GenAI… · p. 8
  • Respondents favored actionable advice to double-check, verify, or ensure results work for them, though some wanted more detailed checking guidance.The authors recommend clear, actionable advice for checking results.Source: Guidance for Disclosures of GenAI… · p. 8
  • Respondents reacted positively to messages that users can flag questionable results and that feedback helps improve GenAI tools.The authors recommend reminding users that they can provide feedback.Source: Guidance for Disclosures of GenAI… · p. 8
  • Respondents favored descriptions of accuracy and responsibility measures, and wanted more detail about training data, creator consent, and environmental costs.Examples respondents reacted positively to included training data, rigorous testing, and privacy and security measures.Source: Guidance for Disclosures of GenAI… · p. 10
  • Respondents favored phrases describing GenAI as constantly improving, getting more accurate, or evolving.The authors suggest explaining that effort is going into improving products and underlying models.Source: Guidance for Disclosures of GenAI… · p. 10

Future Work

  • Future research could track public attitudes as agentic AI evolves and examine whether checking inclinations change as errors become fewer or subtler.The authors report high awareness of GenAI mistakes among users and non-users.Source: Future Work · p. 10
  • Users who verify only when results seem suspicious may miss difficult-to-detect errors, motivating interest in verification tools.The authors caution against placing too much accuracy burden on users.Source: Future Work · p. 10

Related papers

Loading 2609.38186v1…