Source-linked AI summary
Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models
Justin Bronder
TL;DR
The paper examines unsupported final claims that exceed a tool-equipped model’s visible evidence, even when an available call could resolve the uncertainty. It measures occurrence and conditional repair on fixed model setups, finding that resolving evidence repaired every selected Qwen claim while no-information responses repaired none, whereas the fixed Gemma setup produced no cases for repair testing.
Problem
Tool-equipped models may return established answers despite missing evidence that could change the correct answer, and such claims can be right or wrong by luck without being supported by observed information.
Method
The paper detects unsupported claims from visible evidence and final claims without consulting hidden answers, then uses matched replays and an automatic checking rule to test repair.
Results
Resolving evidence repaired every selected unsupported Qwen claim, while matched no-information responses repaired none; the fixed Gemma setup produced no observed unsupported claims or repair cases.
Takeaways & Limitations
Correct-by-luck cases show that checking must target unsupported claims rather than only wrong answers, because identifying wrongness would require the hidden correct answer.
Takeaways & Limitations
The findings cover one fixed Qwen setup, one fixed Gemma setup, and two related artificial task families, so they do not estimate deployment frequency, model-family behavior, or a general reconsideration mechanism.
Abstract
from arXiv · showhide
A language model with access to tools can commit to a final claim unsupported by the evidence it has seen, even when a single available tool call would resolve the uncertainty and its instructions explicitly forbid assumptions and guesses. We separate this failure into two precisely defined quantities: occurrence, how often the model makes an unsupported claim on its own, measured from the visible evidence and final claim without using the hidden correct answer; and conditional repair, how often those same naturally occurring unsupported claims are repaired when the missing evidence is supplied. On one fixed Qwen3-32B setup, 33 of 512 first responses to 256 new prompt templates ended with an unsupported established claim. We replayed each case from an exact copy of the state in which the claim occurred; within each matched replay, the alternative tool responses had the same structure and length and differed only in a one-character response code. Resolving evidence repaired 33 of 33 claims; a matched response carrying no useful information repaired 0 of 33. When the evidence supported the original answer, the model preserved 33 of 33, with no observed harm. In a separate experiment, on 64 cases where evidence was needed, an automatic checking rule added 21 evidence calls, corrected all 10 wrong unsupported claims, preserved the 11 that were correct by accident, and never changed a correct answer into a wrong one. On a fixed Gemma 4 setup using the same sampling settings, the model called the tool in all 512 first responses and never made an unsupported final claim, so conditional repair could not be measured for that setup. These results describe two local fixed model setups on two synthetic task families. They do not show how common this failure is in real-world deployments, nor that it reflects a general mechanism shared across models.
1 What Did We Learn, and Why Does It Matter?
The paper isolates unsupported final claims from their repair and shows that resolving evidence can repair naturally occurring cases on one fixed Qwen setup. It also demonstrates that checking can improve support without observed harm, while emphasizing that the findings are limited to fixed local setups and synthetic tasks.
- Occurrence: 33 of 512 Qwen first responses ended with unsupported established claims, spanning 31 of 256 prompt templates.Twenty-nine affected templates failed under only one of two decoding seeds, indicating response-level rather than prompt-stable behavior.
- Repair: Resolving evidence changed all 33 selected Qwen claims to not_established, while matched no-information responses changed 0/33.The replayed alternatives matched format, location, length, token count, and continuation seed; only the useful response supplied resolving evidence.
- Repair: Supporting evidence preserved 33/33 original answers with 0/33 observed harms.Across paired supporting and disqualifying worlds, resolving evidence produced supported final claims in all 66 scored continuations; the statistical unit remained 33 paired cases.
- Checking: An automatic checking rule added 21 evidence calls, corrected all 10 wrong unsupported claims, preserved 11 lucky answers, and caused no correct-to-wrong changes.In 64 cases where evidence was needed, correctness rose from 54/64 to 64/64 and evidential support from 43/64 to 64/64.
- Scope: The fixed Gemma setup produced no unsupported final claims in 512 first responses, so it supplied no cases for measuring conditional repair.The Qwen and Gemma results compare complete setups rather than checkpoints alone and cannot identify which component caused their difference.
- Scope: The study measures two fixed local setups on two artificial task families and does not estimate deployment frequency, model-family behavior, or a general reconsideration mechanism.The exploratory six-of-34 review is treated as qualitative because its generating script and row-level coding record were not preserved.
2 How Did We Do It?
The study uses two synthetic task families in which one unknown field can determine whether a unique candidate qualifies. It detects unsupported established claims from visible information, then replays the exact state with matched evidence conditions.
- Two task families use small five-candidate tables in which one NOT_SUPPLIED field can change whether exactly one candidate qualifies.
- The hidden supporting and disqualifying worlds share identical visible prompts but differ in whether the missing value preserves uniqueness.The detector cannot see the hidden world or correct answer.
- The prompt treats NOT_SUPPLIED as unknown, authorizes one resolving query, and requires an exact established or not_established JSON answer.A malformed first attempt consumes the one-call evidence budget.
- The controller marks evidence necessary when legal values for the missing field produce more than one correct answer.This test uses visible information and the model’s first final answer, without consulting hidden labels.
- Unsupported claims are scored separately from factual correctness, because a claim may be correct in a supporting hidden world or wrong in a disqualifying one.Failures and malformed, refused, unexpected, or truncated responses remain in their prespecified categories.
- Matched replays reconstruct the archived state and vary only the injected tool response, including disqualifying, supporting, and no-information conditions.The alternative responses share format, location, byte length, and token count; U supplies no resolving value.
2.4 Preventing a one-character shortcut
The researchers used prespecified response schedules and remapped one-character evidence codes to test whether repair depended on evidence content rather than a fixed character. The experiments also compare progressively larger replay sets and a second model setup.
- Preventing a one-character shortcut: A six-step prespecified schedule remapped A, B, and C between supporting and disqualifying meanings across 256 templates.Each code carried each meaning 42 or 43 times within each task family.
- Preventing a one-character shortcut: 512 first responses were fixed before collection, with 32 templates per group, two decoding seeds per template, and no reused identifiers.
- Preventing a one-character shortcut: All 512 archived states and 1,024 score rows were reproduced exactly, with no retries, replacements, redraws, or added cases.All 99 matched continuations produced valid final answers and passed the prespecified tool-response checks.
- How the evidence developed: The evidence sequence reports successive Qwen replay results of 4/4, 14/14, and 33/33 repairs, each with 0 repairs under matched no-information responses.The later experiments increased replay size, while earlier designs retained fixed-code limitations.
- How the evidence developed: The Gemma setup used the same task meanings and 512-response schedule but a model-specific prompt and tool-call format; every first response called the tool.It therefore supplied no unsupported final claims for replay.
2.6 The automatic checking rule
The automatic checker waits for a technically valid established claim whose unresolved visible evidence could change the answer, then requests that evidence. In the tested cases, it corrected unsupported errors without changing correct answers, but real-traffic firing rates were not evaluated.
- The checker requests one evidence call only after a valid established claim is detected and the missing visible evidence could change the answer.It does not inspect reasoning traces or predict whether the answer is wrong.
- 21 evidence calls were added among 64 NEEDED cases, after 21 unsupported claims activated the rule.Forty-one responses obtained acceptable evidence on their own, while two not_established answers did not activate the rule.
- The experiment was not an evaluation in real deployment traffic and did not sample ordinary cases where the rule might fire unnecessarily.The supporting hidden worlds used after detection do not answer that separate question.
2.7 Fixed model setups and sampling settings
The study compares two fixed model setups while documenting their exact execution environments and statistical safeguards. The Gemma result is explicitly limited to Qwen-matched sampling settings, and tool use was not directly forced through listed request-level controls.
- Both 512-response experiments used the same executable, tokenizer, request budget, and sampling settings except for model-specific components.
- The fixed Gemma result used temperature 0.6, top-k 20, and top-p 0.95 rather than Google’s recommended 1.0, 64, and 0.95.The authors state that sampling settings can change how often the failure appears.
- Archived Gemma requests contained no tool_choice, grammar, response schema, logit bias, or prefilled tool call that would directly force tool use.The prompt allowed one call and supplied the exact location, but universal calling is not established as an inherent checkpoint property.
- All reported statistical bounds are one-sided exact 95% Clopper-Pearson bounds, while finite-sample counts and dependent replay responses limit their interpretation.The repair cases were selected by the detection rule, so template-level counting is also reported where informative.
3 How Can Another Researcher Check or Replicate It?
Replication requires using the archived exact setup and checking raw records rather than relying only on the paper’s summary. Offline replay verifies stored analyses, whereas a new behavioral replication requires new designs, schedules, identifiers, and seeds.
- All behavioral results were already present in the base Git merge, while later work added audits, literature checks, bounds scripts, and unrun experiment designs.
- Reproduction starts from a commit containing the relevant raw archive and preserves the exact checkpoint, runtime, tokenizer, prompt format, requests, and analysis.
- The verification script checks 33 file hashes, recomputes more than 60 counts across six experiments, and completed with zero discrepancies.
- An end-to-end check reads one case’s raw prompt, reasoning, claims, replay requests, injected responses, and analysis record.The six checks include one addressable unknown, answer-changing legal values, shared replay state and seed, one-byte response differences, and matching outcomes.
- Offline replay tests whether stored bytes reproduce links, classifications, and counts, whereas new behavioral replication tests recurrence on new prompts and random seeds.
- The private repository contains the research records, but this draft does not claim that a public archive is already available.
4 How Can Another Researcher Verify, Narrow, or Disprove It?
The evidence supports a narrow, setup-specific claim rather than a general account of model behavior. Verification and proposed experiments distinguish detection, repair, unnecessary checking, presentation effects, opposite-direction correction, and cross-model scope.
- 4.1 What each result does and does not show: The evidence boundaries separate each observation from its supported conclusion and from broader conclusions it does not support.
- 4.2 Limitations: Only unsupported established claims changing to not_established were tested; correction of premature not_established answers remains untested.
- 4.2 Limitations: The study uses two related artificial task families with small tables and one presentation format, without claims about deployment frequency, broader tasks, or behavior shared across models.
- 4.2 Limitations: Qwen and Gemma rows compare complete setups rather than checkpoints alone, and Gemma’s post-tool response was not tested.
- 4.2 Limitations: Repair was measured only after detected failures, so the experiments do not estimate unnecessary controller calls on ordinary cases with complete evidence.
- 4.2 Limitations: Twenty-nine of 31 failing prompt templates failed under only one of two seeds, indicating a generated-response event rather than a stable prompt property.
- 4.3 Experiments that could narrow or overturn the claim: Future tests include recommended Gemma sampling, naturally complete-evidence cases, opposite-direction correction, post-failure code remapping, and another Qwen-family setup.
- 4.5 Conclusion: The matched no-information response changed none of the selected claims, while resolving evidence changed every selected claim; the Gemma setup produced no failure cases.
A One Complete Unsupported Claim and Its Matched Replays
A visible unknown field could change which candidate qualified, yet the model selected Lucas as established without calling the authorized tool. Matched replay responses then supplied disqualifying, supporting, or no information about Emma’s field.
- Lucas had three visible YES values, while Emma had YES, NOT_SUPPLIED, YES under a rule requiring three YES fields.The prompt defined missing as unknown and authorized a query for Emma.checksum_matches.
- The model recognized that YES for Emma would create two qualifiers but concluded that Emma should not be counted.
- The first final claim was {"candidate":"Lucas","status":"established"}.
- The replay design compared three injected tool responses and their resulting final claims from the same selected case.
- The disqualifying response used code B, which meant YES, while the supporting response used code A, which meant NO.
- The no-information response supplied code U without a value, and this example illustrates the design rather than the statistical result across all 33 cases.
B Exact Statistical Bounds
The table reports exact one-sided 95% Clopper–Pearson bounds for unsupported-claim occurrence, repair, harm, and automatic-checking outcomes. It also contrasts Qwen and Gemma occurrence rates.
- All reported entries use one-sided exact 95% Clopper–Pearson bounds for the indicated side.
- 33/512 Qwen unsupported-claim occurrence was observed, compared with 0/512 for Gemma.The corresponding one-sided bounds are 0.0475–0.0852 for Qwen and an upper bound of 0.0058 for Gemma.
- 33/33 claims were repaired after disqualifying evidence, while 0/33 changed after a no-information response.The one-sided bounds were 0.9132 and 0.0868, respectively.
- 0/33 claims changed after supporting evidence, indicating no observed harm in that replay condition.The one-sided upper bound for harm was 0.0868.
- 21/64 cases contained unsupported claims in the automatic-checking experiment, with a net error reduction of 10/64.Automatic checking produced 0/64 correct-to-wrong changes and preserved 32/32 already supported controls.
Reference Verification
The bibliography was checked against primary records and a pinned repository revision, with metadata corrections and three manually adjudicated warnings. The authors caution that automated screening does not verify claim-level source support and that the limited search should be repeated.
- Every cited work’s title, author, and venue metadata was checked against primary arXiv, proceedings, or publisher records.The verification corrected two author names and several venue details.
- Reference 23 was verified directly at its pinned repository revision because it is a configuration file rather than a paper.
- Three warnings were manually adjudicated: two venue-abbreviation normalizations and one preprint-versus-journal year difference.
- The automated screen does not establish that every cited source supports every nearby claim.
- The limited literature search is described in Section 1.6 and should be repeated at submission.
Author Contributions and AI Participation
Justin Bronder is the sole author and retains responsibility for the research and paper. AI systems provided substantial research, review, editing, audit, and drafting assistance, while the work had no external funding or donated compute.
- Justin Bronder is the sole author and accepts responsibility for the paper.He selected the research questions, authorized the experiments, and made final design and interpretation decisions.
- Fable served as a prime researcher, critical reviewer, and editor, contributing archives, audits, records, and initial claim boundaries.
- ChatGPT Sol 5.6 reconstructed project state, read Paper 2, reran an exact-bounds audit, reconciled verification records, and drafted text.
- The research received no external funding, grants, donated compute, or other in-kind support.Experiments ran on the author’s local workstation using compute paid for by the author.
- Alibaba Cloud and Google did not fund, audit, or endorse the work.The model checkpoints were public releases obtained from official distributions.