Source-linked AI summary
From the Gettier Machine to Justified True Output: A Three-Step Account of Rational Trust in Generative AI
Tevfik Uyar
TL;DR
The paper asks when trust in generative AI outputs is rational, given that users treat chatbot text differently from reliable tools such as calculators. It proposes acquisition, understanding, and independent verification as the conditions for justified true output: only propositions passing all three steps merit knowledge-like trust.
Problem
The paper asks when a proposition taken from an AI output becomes knowledge and when trust in it is rationally established.
Source: §1 · p. 1Method
The paper maps acquisition, understanding, and independent verification onto belief, justification, and the anti-luck sensitivity condition.
Source: §1 · p. 2, §4.1 · pp. 6–7Results
Only an AI-output proposition for which the user completes all three steps attains justified true output status and merits knowledge-like trust.
Source: §1 · p. 2Takeaways & Limitations
The paper offers a stakes-indexed criterion for rational trust and an everyday test of whether a false output would have been noticed.
Source: §1 · p. 2, §4.1 · p. 7The clock analogy concerns how production volume accumulates observed hits, not the model’s truth ratio; past hits do not establish that a particular output is true.
Source: §5 · p. 9
Abstract
from arXiv · showhide
As generative artificial intelligence (AI) settles into everyday practices of inquiry, an old epistemological question returns in a new guise: when does a proposition a user takes from an AI output become something the user knows, and when is it rational to trust it as such? This paper argues that the characteristic failures of AI-mediated inquiry are precisely those that the justified-true-belief (JTB) tradition and its post-Gettier anti-luck refinements were built to diagnose, and it advances three claims. First, AI outputs inherit neither the presumptive warrant of testimony nor the reliabilist warrant of deterministic tools: unaudited trust in a calculator rests on a type-level reliability record, and for general-purpose generative models there is no type-stable process to which such a record could attach -- a practical form of the generality problem. Second, use that stops at obtaining an output turns generative AI into a Gettier machine: the user's fluency-based seeming of justification is disconnected from the truth of what is believed, and, as a clock running ten times too fast illustrates, sheer speed of production multiplies coincidental hits, inflating the appearance of reliability without improving the standing of any single hit. Third, the remedy is a three-step structure of use -- acquisition, understanding, independent verification -- whose first two steps answer to the classical belief and justification conditions and whose third answers to the post-Gettier sensitivity condition. The result is a stakes-indexed criterion of rational trust, justified true output (JTO), with an everyday test: "Had this been false, would I have noticed?" The criterion is defended against objections from over-demandingness, the extended mind, model improvement, and redundancy.
1 Introduction
The paper asks when trust in generative AI is rational and when a proposition in its output can count as knowledge. It proposes a three-step account that connects AI use to classical and post-Gettier conditions on knowledge.
1 Introduction
- The paper locates the problem in a mismatch between calculator-like trust and unchecked acceptance of fluent AI narratives, asking what makes the latter rational.It frames the question normatively: when does trust in an AI proposition become rational, and when does that proposition count as knowledge for the user?Source: §1 · pp. 1–2
- The proposed sequence is acquisition, understanding, and independent verification, corresponding respectively to belief, justification, and the post-Gettier condition that truth not be accidental relative to the user’s grounds.The user first obtains an output, then reconstructs its reasons, then checks its propositions against sources independent of the model.Source: §1 · p. 2
- Only propositions whose users complete all three steps qualify as justified true output and rationally merit knowledge-like trust.Stopping after acquisition is at best accurate opinion, even when the output happens to be true.Source: §1 · p. 2
- The paper identifies three contributions: explaining why AI lacks testimonial and calculator-like warrant, analyzing user-side Gettier cases and coincidental accuracy, and proposing a stakes-indexed trust criterion.The introduction also previews the paper’s progression through the classical account, AI’s epistemic status, the criterion, Gettier cases, objections, applications, and conclusion.Source: §1 · p. 2
2 The Classical Analysis: Why True Opinion Is Not Knowledge
The classical analysis distinguishes knowledge from true opinion by requiring belief, truth, and justification, but Gettier cases show that these conditions can coincide by luck. Anti-luck accounts add conditions intended to block that coincidence.
2 The Classical Analysis: Why True Opinion Is Not Knowledge
- Plato’s account makes reasons the bond that keeps true opinion from escaping, so knowledge is an achievement attributable to the agent rather than mere outcome accuracy.The paper invokes the traveler who reaches Larissa by a correct guess: the result matches that of someone who knows the road, but the guess is not tethered by reasoning.Source: §2 · p. 3
- The standard JTB account requires belief, truth, and justification, yet Gettier cases show that justified true belief can still fail to count as knowledge when truth and justification coincide accidentally.Post-Gettier proposals include reliable processes, sensitivity, and safety; Pritchard argues for both an anti-luck condition and an ability condition.Source: §2 · p. 3
- The paper uses JTB and anti-luck theories diagnostically rather than attempting to settle their internal disputes, mapping their familiar problems onto AI-mediated inquiry.It connects step-skipping to a lucky guess, fluent prose to a feeling of justification, and a true hallucination to a Gettier case.Source: §2 · p. 3
3 The Epistemic Status of AI Output
The paper rejects treating generative AI as a stored knowledge bank, a human-like witness, or a calculator with stable reliability. It instead describes AI as an epistemic technology whose outputs still require user understanding and oversight.
3.1 Not a Knowledge Bank
- Generative AI regenerates text probabilistically rather than fetching stored information, and the same process can yield true and false claims without an internal feature that distinguishes them.Thus fluency, confidence, or apparent reasons cannot tell a user from the output alone which case they have encountered.Source: §3.1 · p. 4
3.2 Not a Witness
- AI output lacks the speaker responsibility and epistemic conscience that support presumptive trust in human testimony, leaving the epistemic responsibility with the user.Even statement-based or quasi-testimony views do not restore the presumption: the hearer still needs positive reasons, and machine errors and moral standing differ from human testimony.Source: §3.2 · p. 4
3.3 Not a Calculator Either
- Unlike a calculator’s fixed, repeatedly tested process, a generative model has no single type-stable reliability record because its truth ratio varies by domain, topic, and prompt phrasing.This is a practical instance of reliabilism’s generality problem: users typically cannot identify which candidate process type their output instantiates or know its reliability.Source: §3.3 · p. 5
3.4 AI as an Epistemic Technology
- AI is an epistemic technology that performs information-related operations while their epistemic value remains dependent on human oversight.It can accelerate phases of coming to know, but understanding and verification remain the user’s responsibility.Source: §3.4 · p. 6
4 The Three Steps and the Conditions on Knowledge: Correspondence and Criterion
The paper maps acquisition, understanding, and independent verification onto belief, justification, and an anti-luck condition, then uses this mapping to define stakes-indexed rational trust in AI outputs. JTO requires all three steps where a false proposition would matter and otherwise go unnoticed.
4.1 The Epistemological Content of the Steps
- Acquisition supplies belief, understanding supplies model-independent justification, and independent verification makes belief sensitive to truth without guaranteeing it.Verification uses a reliable process independent of the model; if the proposition were false, the user would very likely notice.Source: §4.1 · pp. 6–7
4.2 The Criterion: Justified True Output
- JTO requires acquisition, reconstructed and appropriated reasons, and independent verification in contexts where a false proposition would matter and escape notice.It is a criterion of rational trust, not a definition of knowledge, and does not claim the conditions jointly suffice for knowledge.Source: §4.2 · pp. 7–8
- Understanding and verification are irreducible: understanding detects missing information and weak reasoning, while verification tests truth against external sources.Their relative work varies: simple facts lean more on verification, while arguments, summaries, and interpretations lean more on understanding.Source: §4.2 · p. 8
- The criterion is stakes-indexed: when the user would notice a falsehood anyway, further verification is wasted cost; otherwise, the user must read, understand, and verify.Its practical test is: “Had this been false, would I have noticed?”Source: §4.2 · p. 8
4.3 The Correspondence in Summary
- The framework translates the classical anatomy of knowledge and its anti-luck refinement into a user’s manual for AI use.Table 1 presents the correspondence between knowledge conditions and the three steps of AI use.Source: §4.3 · p. 8, Table 1 · p. 8
5 The Price of Skipping Steps: AI as a Gettier Machine
When users stop at obtaining an AI output, fluency can seem to justify a belief without connecting it to truth, producing a Gettier-like gap. The fast-clock analogy shows how production volume multiplies coincidental hits, while understanding and independent verification address the user’s epistemic position.
5 The Price of Skipping Steps: AI as a Gettier Machine
- A step-one user may have thin, seeming-based justification, but the persuasive appearance of support is disconnected from whether the proposition is true.On a more demanding view, the user has no justification; either way, the belief falls short of knowledge.Source: §5 · p. 9
- 18 times a day, a clock running ten times too fast shows the correct time, but each correct reading remains coincidental rather than epistemically improved.The analogy generalizes: a clock running k times too fast is right 2(k−1) times a day, while the chance of a given glance catching a correct reading stays the same.Source: §5 · p. 9
- Generative AI multiplies coincidental hits through speed, while fluent, confident answers fail to distinguish true outputs from false ones.The fast-clock analogy models hit-count inflation, not models’ truth ratio; remembered hits support only a generic expectation that outputs are often right.Source: §5 · pp. 9–10
- The Gettier-machine problem lies in the user who stops at acquisition, so the proposed remedy is user understanding and independent verification, not model engineering.Together, those steps move belief from persuasive appearance toward reconstructed reasons and truth-sensitive checking.Source: §5 · p. 10
6 Objections and Replies
The replies argue that JTO is not an arbitrary extra burden: ordinary unaudited trust has reliability or accountability underwriting that generative AI lacks. They also defend the three steps as compatible with extended cognition, responsive to model improvement, and consistent with AI’s useful role in discovery.
6.1 The Over-Demandingness Objection
- Unaudited trust in calculators, encyclopedias, and witnesses rests on reliability records or institutional and social accountability that generative models lack.Because model reliability varies by domain and its outputs lack editorial control or assertoric responsibility, JTO places the burden on the user, scaled to the stakes.Source: §6.1 · pp. 10–11
6.2 The Extended-Mind Objection
- Unlike Otto’s notebook, a model’s query-time output was not previously endorsed by the user; understanding and verification supply the active engagement needed for cognitive extension.The cited Digital Andy analysis leaves open that more agentic, actively integrated architectures might qualify as extensions.Source: §6.2 · p. 11
6.3 The “Models Are Improving” Objection
- A falling error rate does not remove the need for external audit: outputs remain internally indistinguishable as true or false, absent a type-stable reliability record.Independent benchmarks, evaluation, and expert oversight are needed even to establish that models are improving; task-specific reliability records could lighten audit burdens. The passage also describes a self-feeding loop between attributed infallibility and delegated authority.Source: §6.3 · p. 12
6.4 The Redundancy Objection
- AI contributes chiefly to discovery by proposing answers and surfacing reasons; checking can be cheaper than finding, and users’ reconstructed reasons can justify on their own merits.JTO names the status an output gains for the user after the user completes the relevant epistemic work, not a status it has on arrival.Source: §6.4 · p. 12
7 Where the Framework Applies
The framework applies from learning to professional practice and everyday decisions, with its normative burden scaling with the harm a false output could cause. Its steps function as a diagnostic, professional norm, and self-audit.
Where the Framework Applies
- In learning, S2 is where students reconstruct reasons; delegating it delegates learning, so AI-literacy education can center on asking which step the student is taking.The framework also diagnoses why prohibition can fail: it drives use underground, where it is necessarily stepless.Source: §7 · p. 13
- When AI-mediated reports affect others, professionals must verify outputs, retain responsibility, and treat AI as an assistant leading to sources rather than as a source.For journalists, the three steps translate into professional obligations; the passage extends this application to lawyers, physicians, engineers, and managers as stakes grow.Source: §7 · p. 13
- For everyday decisions, “Had this been false, would I have noticed?” helps users determine whether their context already satisfies the sensitivity condition or further steps remain.The framework presents this as a self-audit for deciding which uses demand which steps.Source: §7 · p. 13
8 Conclusion
The conclusion maps acquisition, understanding, and verification onto belief, justification, and non-accidental truth, making user verification central to rational trust in AI outputs. It warns that skipped steps leave users with lucky opinion and identifies stakes for individual and collective reliance.
8 Conclusion
- Acquisition, understanding, and verification are practical counterparts of belief, justification, and truth that is not accidental relative to one’s grounds.JTO measures whether users have shouldered the burden of justification and verification; its everyday test asks whether they would have noticed if the output were false.Source: §8 · p. 14
- A user who stops at acquisition has, at best, true opinion whose accuracy is not owed to the user’s epistemic achievement.The passage compares this luck to a clock running a thousand times too fast.Source: §8 · p. 14
- The paper argues that abandoning the steps risks loss of learning and knowledge individually and, collectively, unexamined decisions and erosion of critical thinking and human agency.It describes the collective consequence as “technocratic theocracy,” following Uyar (2024).Source: §8 · p. 14