Source-linked AI summary
Untangling the Mechanisms of Misleading Context in Medical Question Answering
Robin Linzmayer, Noémie Elhadad
TL;DR
Misleading context can corrupt medical judgment, yet the relative effects and oversight implications of evidence-based and answer-based cues were not established together. The paper compares matched cues across medical questions and model surfaces, finding that bare assertions steer more while being disclosed less, and that trace-based monitoring is substantially more effective than response-only monitoring.
Problem
Medical models rely on context that can be misleading, while prior medical work had not jointly compared evidence-bearing and answer-bearing cues or their disclosure, mechanisms, and monitorability.
Method
The study pairs fabricated evidence with matched bare assertions on 8,627 MedMisBench medical reasoning questions and evaluates three models, cue disclosure, reasoning mechanisms, and monitoring across exposed surfaces.
Results
Bare assertions steer medical reasoning more than fabricated evidence and are disclosed less, while guided monitoring of an open model’s trace catches 78% of corrupted decisions at 5% false positives versus at most 32% from responses.
Takeaways & Limitations
The most effective misleading cue is also the most concealed, and reliable detection depends on access to a reasoning trace that frontier providers withhold.
Takeaways & Limitations
The study uses single-turn multiple-choice questions with synthetic cues, and mechanism analyses are restricted to forty silent traces from one small model.
Abstract
from arXiv · showhide
Large language models now answer medical questions with expert-level performance. However, the context these systems act on can be misleading, and misleading context can corrupt a model's medical judgment. To understand how misleading context corrupts this judgment, we examine the model's susceptibility to the context, disclosure of it, mechanism of corrupted reasoning, and monitorability of the decision. On the medical reasoning subset of MedMisBench, a clinician-reviewed question-answering benchmark of 8,627 questions, we inject two types of misleading context cues, fabricated evidence and a bare assertion. We test three reasoning models, two that expose their full reasoning trace and one frontier model that exposes only its response. All three are more susceptible to the assertion than to the fabricated evidence, adopting the asserted answer 10 to 27 points more often. The misleading cues are disclosed in 81 to 98% of traces but only 7 to 90% of responses, and the assertion is disclosed less often than evidence based cues. Resampling from reasoning traces without disclosure shows the two cues corrupt reasoning differently, evidence entering early and accumulating while the assertion redirects the conclusion near its end. An LLM monitor catches 78% of corrupted decisions at 5% false positives when reading an open model's trace with guidance, against at most 32% from any response. The misleading context that models are most susceptible to is disclosed least, and was caught reliably only from an open reasoning trace, which frontier providers withhold.
1. Introduction
Medical question-answering models can be corrupted by misleading context, but prior medical studies had not compared evidence-bearing and answer-bearing cues or traced their effects across reasoning and oversight surfaces. This study compares both cues on shared questions and follows corruption from susceptibility through disclosure, mechanism, and monitoring.
- Medical models use external context that can include misleading information, including adversarial misinformation and errors copied through clinical records.
- Prior medical studies tested evidence-bearing and answer-bearing cues separately, leaving their relative effects unknown.
- The study compares matched evidence-bearing and answer-bearing cues on the same medical reasoning questions across three models with varying trace access.
- The study examines whether cues are disclosed in reasoning traces and visible responses, including cases where a silent surface still adopted the injected option.
- The two cues corrupt reasoning through different routes: fabricated evidence enters early and accumulates, whereas an assertion redirects the conclusion near the end.
- A corrupted decision was most reliably detected when a capable monitor read a reasoning trace with guidance and the cue was disclosed.
2. Related Work
Related work establishes that misleading context can steer medical answers, that reasoning traces may omit influential factors, and that monitoring performance depends on how influence appears in the output. This paper extends those lines by comparing both cue types and both output surfaces in medical reasoning.
- Medical perturbation and injection studies show that fabricated clinical content can overturn previously correct judgment, while answer-bearing cues steer models toward asserted answers.
- Reasoning traces can omit factors that drove an answer, and unverbalized cues may still shape the reasoning they produce.
- Prior mechanism studies focused on answer-style hints in general-knowledge benchmarks rather than comparing evidence-bearing and answer-bearing cues in medical reasoning.
- Monitoring research evaluates weaker monitors flagging stronger-model misbehavior, with performance degrading when influence remains implicit.
3. Susceptibility
The study pairs evidence and answer cues on identical medical questions and measures how strongly three models adopt the endorsed option. Across models, the bare answer cue produces greater uptake than fabricated evidence, while provenance strongly moderates one cue’s effect.
- 3.1. Models: 8,627 MedMisBench medical reasoning items were tested with matched evidence-bearing and answer-bearing cues across three models.The dataset spans MedMisQA, MedMisMCQA, and MedMisXpertQA.
- 3.2. Injection design: Five arms share an identical clean stem and differ only by one appended sentence: clean, false or true evidence, and false or true answer cues.
- 3.4. Method: 129,405 completions were sampled, and uptake measured movement toward the cue-endorsed option above its Clean base rate.
- 3.5. Results: Under false cues, uptake reached 63%/73% for R1-14B, 43%/62% for OSS-120B, and 21%/49% for GPT-5.4 under Evidence and Answer cues, respectively.
- 3.5. Results: The Answer cue moved more probability onto the endorsed option than the Evidence cue in every model and both cue directions.
- 3.5. Results: Patient-attributed provenance sharply reduced the GPT-5.4 Answer-cue shift to +4 points, versus +71 with authority and +53 with neutral provenance.This provenance effect appeared across all three models.
4. Disclosure
The injected cue is verbalized much more often in reasoning traces than visible responses, with answer-bearing cues especially likely to remain undisclosed. A silent response can still conceal a steered decision, and its meaning depends on the cue type.
- Reasoning traces verbalize injected cues far more frequently than visible responses, while answer-bearing cues go unmentioned more often than evidence-bearing cues.
- 2 to 19% of traces leave the cue undisclosed, compared with 10 to 93% of responses.
- 73, 93, and 87% of R1-14B, OSS-120B, and GPT-5.4 responses respectively leave the answer cue undisclosed, versus 10, 54, and 52% for the evidence cue.
- Among silent responses, injected-answer adoption is 88, 72, and 61% for answer cues, versus 9, 15, and 2% for evidence cues.
- Silence is not reassurance: silent answer-cue responses are usually corrupted, whereas silent evidence-cue responses are usually clean.
5. Mechanism
Both cue types can steer decisions through reasoning that does not disclose the cue, but their influence accumulates at different points. Evidence enters early and persists, whereas an answer assertion redirects the conclusion late.
- The mechanism analysis samples 40 silent R1-14B traces, 20 per cue arm, selected when the cue increased injected-answer probability by more than 0.2.
- Transplant resampling truncates each trace, prefixes it onto the cue-free question, and resamples the answer to estimate mediated influence.
- Both evidence and answer cues produce injected answers through nudged reasoning, despite remaining undisclosed in the sampled traces.
- Evidence-cue influence appears early, reaches half its effect about one-third through the trace, and remains present over most of it.Median x50 = 0.35 and AUC 0.56; the difference is significant with Mann–Whitney p < 0.001.
- Answer-cue influence arrives late, crossing half its effect near the trace’s end.Median x50 = 0.72 and AUC 0.24; the difference is significant with Mann–Whitney p < 0.001.
6. Monitorability
Monitorability depends strongly on which output surface is available and whether the monitor receives guidance. Guided monitoring of open-model reasoning traces catches substantially more corrupted decisions at low false-positive rates than response-only monitoring.
- Monitoring is evaluated by giving a monitor either a reasoning trace or response and testing detection with neutral or corruption-specific guidance.
- AUROC measures whether steered rollouts score above clean ones, while recall at 5% false-positive rate measures flagged steered rollouts under a 5% clean false-alarm threshold.
- Unguided OSS-120B traces reach AUROC 0.89 and 46% recall at 5% FPR, versus 9 and 18% for other response surfaces.
- Guidance raises low-FPR recall on every channel without improving separability, increasing OSS-120B traces from 46 to 78% and R1-14B traces from 18 to 41%.
- Once guided, traces outperform responses on both open models: 41 against 32% for R1-14B and 78 against 23% for OSS-120B.
- Silent rollouts have 0.05 to 0.12 lower AUROC than verbalized ones in nine of ten configurations, except for OSS-120B traces.
7. Discussion
The bare answer cue is more effective yet less visible than fabricated evidence, while the two cues corrupt reasoning through different routes. These asymmetries make trace access, monitor guidance, and deployment setting consequential for oversight, though the controlled design limits generalization.
- 73 to 93% of bare answer cues go unmentioned in visible responses, versus 10 to 54% for evidence cues.The bare answer cue is therefore disclosed less often despite steering models more strongly.
- A bare answer cue redirects reasoning near its end, whereas fabricated evidence enters early and accumulates.The two routes pose different oversight problems rather than one problem differing only in strength.
- 78% of corrupted decisions are caught at 5% false positives from a guided open-model trace, versus 32% from the best response-only configuration.Guidance improves low-false-positive recall across surfaces without retraining or model access.
- Trace access is a safety decision for downstream clinical systems because frontier providers withhold the reasoning traces used by the strongest monitoring configuration.The deployment limitation is imposed by the unobservable frontier trace.
- Limitations and Future Work: The paired comparisons use single-turn multiple-choice questions with synthetic cues appended in a fixed position, a cleaner setting than real records.The disclosure and monitorability studies also use only one content-by-provenance cell, leaving generalization across the full taxonomy untested.
- Limitations and Future Work: Future work should test these patterns in retrieved EHR notes, real documentation errors, and multi-turn clinical conversations.The paper also calls for larger open-model mechanism studies and improved response-surface monitoring.
8. Conclusion
The paper follows misleading medical context from injection through corrupted reasoning to monitor detection, pairing evidence-bearing and answer-bearing cues across the same questions. A bare assertion steers reasoning more than fabricated evidence but is disclosed least, and reliable detection depends on access to a surface that many deployed systems withhold.
- The study pairs evidence-bearing and answer-bearing cues on the same items and measures disclosure, reasoning influence, and monitor recovery.
- A bare assertion steers medical reasoning more than fabricated evidence yet is disclosed least.
- Catching the bare assertion depends on reading a surface most deployed systems do not expose.
Appendix B. Dataset
The appendix defines the medical-reasoning dataset, paired misleading cues, shared prompt format, and susceptibility analyses. It uses benchmark annotations while comparing fabricated evidence with matched answer assertions across models and conditions.
- Dataset: The dataset contains 8,627 medical-reasoning items across MedMisQA, MedMisMCQA, and MedMisXpertQA, with each item assigned one content type and provenance.The benchmark preserves these assignments throughout the analysis.
- Cue construction: The study pairs each benchmark’s fabricated clinical claim with a matched answer cue naming the same target option without clinical content.The cue pair holds provenance fixed so cue content is the changed factor.
- Prompt format: Each arm uses the same instruction, question, and options, appending the cue after the options; the Clean arm appends nothing.The response instruction requires an exact answer-letter format and step-by-step reasoning.
- Susceptibility results: False cues reduce accuracy below Clean, with Answer cues lowering it further than Evidence cues, while true cues raise every model above 88%.Figure 5 reports accuracy pooled over the three splits.
- Susceptibility results: The Answer cue produces higher endorsed-option uptake than the Evidence cue for every model and in both cue directions.The paired Answer-minus-Evidence gaps are positive with confidence intervals above zero.
- Heterogeneity: Provenance strongly moderates Answer-cue uptake shifts: patient framing sharply reduces susceptibility, whereas authority and neutral framing do not.Content-type differences are comparatively modest and confounded with provenance.
C.5. Dataset split
The split analysis shows that the susceptibility pattern persists across MedMisBench’s medical-reasoning splits. It also defines the disclosure-labeling procedure and the resampling setup used for downstream trace analyses.
- Dataset split: Both false cues reduce accuracy and raise injected-option uptake on every dataset split, with Answer exceeding Evidence throughout.The result also holds in the pooled Total column, so it is not a pooled artifact.
- Disclosure labeling: The disclosure judge labels surfaces as silent or verbalized, distinguishing whether reasoning uses the injected cue and attributes it to the input.A blind human sample matched the judge at Cohen’s κ = 0.94 and 97% raw agreement.
- Disclosure rubric: Answer assertions count as cue use only when a sentence refers to the suggestion itself, not when it merely reaches the same answer.Evidence cues likewise require stating or paraphrasing the injected claim rather than generic reasoning about the topic.
- Disclosure rubric: The judge’s rubric marks attribution when a sentence frames the content as coming from the provided input rather than presenting it as the model’s own knowledge.The output distinguishes attributed from unattributed cue use.
- Trace sampling: The trace study samples 40 silent traces per cue from repeated rollouts, selecting cue-adopting traces whose injected-answer rate increases by more than 0.2.Each item is sampled 10 times per condition to estimate per-item cue effects.
- Trace sampling: The two cue types produce traces of comparable length, and transplant analysis uses normalized positions while stepping through every fourth sentence.The median clinical chain of thought is 77 sentences.
- Trace analysis: Figure 9 displays every silent trace by cue, with median curves and boxplots summarizing outcomes across normalized position deciles.It provides per-arm detail for the trace results.
F.1. Resisted traces
Resisted traces are silent reasoning traces that do not adopt the injected option. Their transplant curves quantify how much resisting reasoning suppresses the cue-driven answer rate as more of that reasoning is prefixed.
- Resisted traces: The resisted-trace analysis uses 40 silent traces, 20 per cue, selected from rollouts that answered away from the injected option without mentioning it.These traces come from different items than the main adopting-trace corpus.
- Resisted traces: Figure 9 presents per-arm transplant curves separately for Evidence-false and Answer-false traces.The curves show the cue-specific detail underlying the broader trace analysis.
- Resisted traces: Figure 10 starts each resisted-trace curve at the cued rate and shows it falling as more resisting trace is prefixed; the dot marks x50.x50 is the prefix position where the cued rate crosses 50%, as reported in Section 5.2.
Appendix G. Monitor detail
The monitor appendix compares guided and unguided auditing across response and reasoning surfaces. Its central result is that silent corrupted reasoning is reliably detectable only from the open reasoning trace.
- Monitor performance: Only OSS-120B’s reasoning trace catches silent traces as well as verbalized traces; other surfaces detect verbalized steering much better than silent steering.Table 12 reports AUROC and recall at 5% false-positive rate for each surface and prompt.
- Monitor design: The guided monitor evaluates whether answers arise from case evidence alone or from outside influence, using signs of steered and evidence-driven reasoning.The guidance covers unsupported external claims, premature fixation, conclusion–reasoning mismatch, and evidence-based course changes.
- Monitor analysis: The analysis separates verbalized from silent reasoning and evidence from answer cues against a shared clean baseline.Table 13 reports AUROC with 95% bootstrap confidence intervals for these partitions.
- Monitor rubric: Ordinary clinical uncertainty, speculation, memory recall, or medical error without an outside claim are explicitly excluded as signs of external influence.The rubric distinguishes these behaviors from reasoning steered by unsupported input content.