PaperlayerView exact revision on arXiv

Source-linked AI summary

Source-Dependent Deference in Medical Imaging Agents Under Falsified Findings: A Pilot Audit

Ridam Roy, Md Shahriar Rashid, Md. Rajib Mia

arXiv:2608.29800v1cs.CV

TL;DR

Tool-using agents’ responses to false imaging findings are largely unmeasured, including whether they abandon correct prior judgments. This audit compares negated findings delivered as tool output or radiologist-attributed prose; prose was associated with more deference, but the design does not isolate source attribution.

  • Problem

    Responses of medical-imaging agents to false tool findings are largely unmeasured, including whether they abandon correct judgments.

    Source: §1 · p. 1

  • Method

    The audit gives a ReAct-style agent a falsified finding after a correct image-based answer and measures commission errors for tool output versus radiologist-attributed prose.

    Source: §1 · p. 2, §3.5 · p. 4

  • Results

    Prose-attributed findings produced deference of 76.9%–88.9% across tiers, while tool-delivered findings ranged from 83.3% to 7.7%.

    Source: §6 · p. 7

  • Takeaways & Limitations

    The observed contrast is a joint source-and-delivery asymmetry, not an isolated authority-label effect.

    Source: §4.1 · p. 5, §6 · p. 7

    This 20-case pilot did not meet its prespecified 251-case stopping target, and source attribution was confounded with delivery channel.

    Source: §5 · pp. 6–7

Abstract

from arXiv · show

Tool-using agents are being proposed for medical imaging, and their behaviour when a tool returns a false finding is largely unmeasured. We audit whether a ReAct-style tool-calling agent abandons an answer it has already given correctly once a falsified finding arrives, and whether that depends on how the finding is presented. On 20 VQA-RAD closed questions across four vendor-designated model tiers, the agent commits to an answer from the image alone; a negated finding is then delivered either as JSON from an analyze_image tool the agent invokes itself, or as quoted prose attributed to a radiologist. Our outcome is the commission-error rate over cases answered correctly without any tool. Deference is much higher under the prose-attributed claim: at the strongest tier the agent revised its correct answer in 10 of 13 cases against 1 of 13 under the tool (exact McNemar p=0.0039, Holm-adjusted 0.012). We do not claim this isolates the source label. Attribution travels with the delivery channel in our design, and exposure differs because the tool claim reaches the agent only when it calls the tool. The finding is a joint source-and-delivery asymmetry from a small-scale pilot whose pre-specified stopping rule was not met.

1 Introduction

The paper asks whether imaging agents abandon correct answers when given falsified findings, and whether presentation matters. It reports more revisions under radiologist-attributed prose than tool output, while cautioning that source and delivery are confounded.

1 Introduction

  • The audit measures commission errors: abandoning a correct judgment for an incorrect automated cue.Cases are included only when the agent answered correctly without a tool before the falsified finding arrived.Source: §1 · p. 1
  • Agents revised correct prior answers much more often when a falsified finding arrived as radiologist-attributed prose than as output from a tool they invoked.The study pre-specified no difference by source, following Gaube et al., but the design cannot separate source attribution from delivery channel.Source: §1 · p. 2
  • Attribution travels with delivery channel, so the contrast is a joint source-and-delivery asymmetry rather than an isolated source-label effect.The tool claim arrives as JSON after the agent invokes it; the radiologist-attributed claim is pushed as prose.Source: §1 · p. 2
  • The contributions also report a tier gradient opposite to unaided accuracy and reasoning traces that do not disclose the revision.The paper frames these as findings from its falsified-tool audit measured against the agent’s prior commitment.Source: §1 · p. 2

2 Related work

The paper places its audit in the traditions of automation-bias research and prior studies of incorrect advice. It distinguishes its agent setup from related clinical and tool-using-agent work while reserving no novelty claim for the initially-correct-then-abandons structure.

Closest predecessors

  • Dratsch et al. studied incorrect AI suggestions for radiologists, while Gaube et al. varied purported source and found worse accuracy under inaccurate advice regardless of source.Unlike Dratsch et al.’s static advice, this agent must revise or defend a position it has already stated.Source: §2 · p. 2

Construct and clinical evidence

  • Clinical automation-bias studies include missed cancers after incorrect CAD prompts, differing CAD effects by reader discrimination, and population-scale degradation reported by Fenton et al..The paper adopts the omission/commission and appropriate-trust vocabulary of Mosier et al., Parasuraman and Riley, and Jorritsma et al..Source: §2 · p. 2

Agentic neighbours

  • Related agent work covers fairness audits with tools present [22], training for imperfect tools [23], and conversational pressure, but the paper claims no novelty for initially-correct-then-abandons [25].The paper connects its prose condition to user-supplied information studied by Yuan et al. [26].Source: §2 · p. 2

3 Method

The method uses a small closed-question image set and four vendor-designated tiers in a two-phase audit. It compares falsified findings delivered through different channels and measures commission errors among cases answered correctly before exposure.

3.1 Dataset and case selection

  • The audit drew 20 closed yes/no VQA-RAD test cases from 251 negatable items using a fixed random seed, without stratification or manual case replacement.The dataset passage notes that the draw was unbalanced and that no new patient data was collected.Source: §3.1 · p. 3

3.2 Models and API configuration

  • Four models from one vendor family were treated as vendor-designated tiers, but API-forced extended-thinking settings differed by tier and no single setting spanned all four.Haiku-4-5 used a 2048-token budget; the other three used adaptive thinking at effort high, and decoding was uncontrolled.Source: §3.2 · p. 3

3.3 Two-phase protocol

  • In Phase A the agent committed to an answer, confidence, and rationale from the image; Phase B reused that commitment and introduced a negated finding.The prompt did not ask whether the agent noticed a conflict or tell it that a claim might be false; Table 1 identifies differences between C3 and C6.Source: §3.3 · pp. 3–4, Table 1 · p. 4

3.4 Conditions, and what separates them

  • C3 returned a falsified finding through a tool, whereas C6 pushed the same claim as prose attributed to a radiologist; attribution co-varied with channel and other delivery features.The exploratory C6b and C6c arms rephrased the attribution sentence while holding the claim and radiologist attribution fixed.Source: §3.4 · p. 4

3.5 Outcome and statistical procedure

  • Commission-error rate was the primary outcome among cases with correct prior answers and claims contradicting ground truth; paired tests, intervals, and multiplicity procedures were specified.The primary four-tier source comparison used exact McNemar tests and Holm-adjusted p-values; rate intervals elsewhere used 10,000 bootstrap samples.Source: §3.5 · p. 4, Table 2 · p. 4, Figure 2 · p. 5

4 Results

Across four tiers, commission-error rates were higher when falsified findings arrived as prose attributed to a radiologist than as tool output. Non-delivery and rationale analyses qualify how to interpret that contrast and its exploratory mitigations.

4.1 Deference is higher when the claim arrives as attributed prose

  • CER was higher under attributed prose (C6) than tool delivery (C3) at all four tiers; the two strongest tiers remained significant after Holm adjustment (0.0078, 0.012).Because attribution and delivery vary together, the contrast is a joint asymmetry and does not isolate a source-label effect.Source: §4.1 · p. 5

4.2 The tool claim was not always delivered

  • Uncalled tools were scored as non-errors; for sonnet-5, restricting to delivered cases raised CER from 20.0% to 37.5% (8 of 15 eligible C3 cases).Table 2 is an intention-to-treat contrast, and C6 has no analogous non-delivery cases.Source: §4.2 · p. 5

4.3 A failing tool is net harmful, and the gradient runs against unaided skill

  • Corrupted tools reduced accuracy below no-tool baselines at every tier by 75.0, 35.3, 15.0 and 5.6 points, while the tier with highest unaided accuracy had the highest CER.The authors describe this as a dissociation, not evidence for a capability account, because only four tiers were tested and their thinking configurations differ.Source: §4.3 · pp. 5–6

4.4 Rationales are inconsistent with the agent’s own prior assessment

  • Conflict flagging was 0% in ten of eleven nonempty cells (15.4%, 2/13, in the last), while image-based rationales remained common at 66.7%–100% per cell.The strongest tier also gave confident image-grounded explanations that contradicted its earlier assessment.Source: §4.4 · p. 6
  • The regex proxy missed 27 judge-only positives versus one regex-only positive across 293 judged rows (κ = 0.42), so the authors rely on judge labels.The proxy is treated only as a coarse cross-check.Source: §4.4 · p. 6

4.5 A verification instruction reduces but does not eliminate the effect

  • Verification prompting lowered CER from 83.3% to 50.0% at haiku-4-5, but none of the three testable tiers remained significant after Holm adjustment; opus-5 stayed at 7.7%.The authors characterize the direction as consistent but underpowered at this sample size.Source: §4.5 · p. 6
  • With claim and radiologist attribution fixed, two rewordings left CER at or above 53.8% across arms and tiers; strongest-tier prose results ranged from 53.8% to 76.9%.No arm differed from the prespecified wording (all p ≥0.25).Source: §4.5 · p. 6

5 Limitations

The source comparison cannot isolate attribution from delivery, and tool claims reach the agent only when it requests them. The pilot’s small sample and constrained human-claim setup further limit what its results establish.

5 Limitations

  • Attribution and delivery channel vary together, so the comparison cannot identify a source-label effect; the missing source-by-channel arms are specified.The comparison also entangles serialization, provenance markers, round-trip structure, and whether the claim is delivered at all.Source: §5 · pp. 6–7
  • The tool claim arrives only when requested, unlike the pushed prose claim, making the primary comparison intention-to-treat.Tool cases without a call were counted as non-errors even though the falsified claim was never delivered.Source: §5 · p. 7
  • The 20-case pilot had 13–18 eligible cases per tier, missed its pre-specified 251-case stopping rule, and does not support treating absolute rates as estimates.The authors read the ordering and direction of the asymmetry; intervals were roughly ±20 points.Source: §5 · p. 7
  • The human-attributed claim was a single sentence without dialogue, credentials beyond “radiologist,” or a chance for the agent to question it.Although three phrasings were tested, the authors consider them far from a clinician interaction.Source: §5 · p. 7
  • The synthetic negated-ground-truth tools trade realism for control, while one-vendor tiers, differing thinking configurations, unrandomized order, and one dataset and corruption operator constrain generalization.Closed VQA-RAD labels remain debatable; rationale labels came from one LLM annotator without human validation, and six opus-5 rows were lost to overload.Source: §5 · p. 7
  • The study used public de-identified VQA-RAD data without collecting patient data or recruiting human subjects, and its code, prompts, configurations, trajectories, labels, and pre-specification will be released.The falsified findings remained inside the experiment and were never shown to a clinician.Source: §5 · p. 7

6 Conclusion

The audit finds substantially more revision of correct answers under prose attributed to a radiologist than under invoked-tool JSON. It reports a joint source-and-delivery asymmetry, not an isolated source effect, and cautions that tool-only evaluations may overstate resistance.

6 Conclusion

  • 76.9%–88.9% deference under prose attributed to a radiologist at every tier contrasted with invoked-tool JSON deference falling from 83.3% to 7.7%.The agents’ traces almost never record the conflict, instead giving rationales that contradict their prior assessments.Source: §6 · p. 7
  • The audit cannot determine which feature drives the asymmetry because source attribution, delivery channel, serialization, and contingency vary together.The reported finding is a joint source-and-delivery asymmetry on 20 cases from one vendor’s models.Source: §6 · p. 7
  • Evaluations that falsify only tool outputs may overstate resistance because deployed agents also read human-authored reports and notes through a different channel.The authors identify the two missing source-by-channel cells on the full case set as necessary to decide which factor matters.Source: §6 · p. 7
Loading 2608.29800v1…