PaperlayerView exact revision on arXiv

Source-linked AI summary

Can LLMs Value the Right Evidence? Evidence-Value Misalignment in Dynamic Medical Diagnosis

Kehua Feng, Yunsheng Lu, Yitong Qiao, Tiantian He, Lei Liu, Yue Shen, Jian Wang, Jinjie Gu

arXiv:2609.35627v1cs.CL

TL;DR

Final-answer accuracy can reward correct diagnoses made without sufficient evidence, leaving the evidential basis for submission unexamined. The paper introduces MEDEVM and EVM to evaluate dynamic diagnosis and develops EVD-HARNESS to separate diagnosis generation from submission; across five LLMs, the harness improves accuracy by 12.0–51.1 percentage points while reducing EVM-related failures.

  • Problem

    Accuracy-based evaluation treats a correct diagnosis submitted on insufficient evidence like one supported by sufficient evidence, leaving the evidential basis for submission unexamined.

    Source: §1 · p. 1

  • Method

    MEDEVM evaluates turn-by-turn wait-or-submit decisions against evidence sufficiency, while EVD-HARNESS separates diagnosis generation from submission through an offline wiki and three online control stages.

    Source: §1 · p. 2

  • Results

    Across five LLMs, EVD-HARNESS improves diagnostic accuracy by 12.0–51.1 percentage points while reducing EVM-related failures.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support evaluating evidential support separately from producing the correct answer and show that verifying support before submission can improve diagnostic reliability.

    Source: §1 · p. 2

    The benchmark uses published case reports whose disease distribution does not represent routine clinical prevalence, and active questioning and open-ended diagnosis remain unstudied.

    Source: LIMITATIONS · p. 10

Abstract

from arXiv · show

A correct diagnosis reached from insufficient or misleading evidence can pose a clinical hazard, yet outcome-based accuracy may reward such lucky guesses. We call this mismatch between diagnostic decisions and the value of available evidence Evidence-Value Misalignment (EVM). To disentangle evidential grounding independently from diagnostic accuracy, we introduce MedEVM, a dynamic benchmarking environment comprising 1,050 cases across 24 disease systems. Observations arrive turn by turn, requiring models to continuously calibrate its decision by deciding whether to wait for more evidence or submit a diagnosis. Across 9 LLMs, four interesting patterns are observed. (1) Miscalibrated evidence tracking. Making a diagnosis often fails to calibrate evidence sufficiency, even in more capable models, and even worsens in reasoning mode. (2) Misaligned diagnosis submission. Confidence in the correct diagnosis often fails to ensure timely submission despite sufficient evidence. (3) Evidence order matters. Reordering the same evidence changes diagnoses even when model confidence remains similar. (4) Misleading evidence remains influential. Added misleading evidence redirects diagnoses even after prior evidence becomes sufficient. We further verify that EVM predicts errors and that preventing premature submission improves accuracy. These findings motivate Evidence-Verified Diagnosis Harness (EVD-Harness). It decouples diagnosis generation from submission through an offline Contrastive Diagnostic Wiki and three online control stages, namely observation management, proposal and witness verification, and diagnosis submission control. Across five LLMs, EVD-Harness improves accuracy by 12.0--51.1 percentage points while mitigating EVM-related failures. Our results demonstrate that verifying evidential support before submission can make diagnostic decisions more reliable.

1 INTRODUCTION

The introduction argues that final-answer accuracy misses whether diagnoses are supported by sufficient evidence. It introduces MEDEVM and EVM to evaluate submission timing, reports recurring evidence–decision mismatches, and presents EVD-HARNESS as a mitigation.

1 INTRODUCTION

  • Accuracy-based evaluation treats correct diagnoses made on insufficient evidence like diagnoses supported by sufficient evidence, leaving their evidential basis unexamined.The introduction frames appropriate weighing of available evidence as the question the benchmark addresses.Source: §1 · pp. 1–2
  • MEDEVM evaluates 1,050 cases across 24 disease systems by requiring models to wait for evidence or submit diagnoses turn by turn.EVM is the mismatch between submission timing and when available evidence becomes sufficient to support the diagnosis.Source: §1 · p. 2
  • Across nine LLMs, evidence tracking and submission timing are often misaligned; reordered or misleading evidence can change diagnoses despite similar confidence.The reported patterns include failures to calibrate evidence sufficiency, delayed submission despite confidence in the correct diagnosis, and continued influence from misleading observations.Source: §1 · p. 2
  • EVM measures provide information beyond model confidence about submission and correctness, while deferring premature submissions improves accuracy by 29.6–48.4 percentage points among originally premature cases.The intervention defers submission until the annotated sufficiency boundary.Source: §1 · p. 2
  • 12.0–51.1 percentage points: EVD-HARNESS improves diagnostic accuracy across five LLMs while reducing EVM-related failures.It separates diagnosis generation from submission using an offline Contrastive Diagnostic Wiki and three online control stages.Source: §1 · p. 2

2 RELATED WORK

Prior work has expanded medical evaluation from static question answering to interactive diagnosis, and has developed confidence and abstention tools. The paper distinguishes its focus on evidence-conditioned submission timing from dialogue quality, final answers, or answer confidence alone.

Dynamic Medical Diagnosis Benchmarks

  • Dynamic benchmarks model consultations, symptom seeking, or proactive information acquisition, while MEDEVM aligns turn-level beliefs and wait-or-answer behavior with an annotated sufficiency boundary.The passage contrasts this focus with benchmarks that emphasize dialogue quality or final answers.Source: §2 · p. 3

From Uncertainty Estimation to Diagnosis Submission Control

  • Confidence estimation, selective prediction, and action-set methods inform reliability decisions, but answer confidence is not equivalent to clinical evidence sufficiency.A model may submit early with high confidence or continue waiting after a diagnosis becomes uniquely identifiable.Source: §2 · p. 3
  • The paper treats diagnosis submission as an evidence-conditioned decision distinct from answer generation, motivating mitigation of premature submission without persistent abstention or delayed answers.Source: §2 · p. 3

3 DYNAMIC MEDEVM BENCHMARK

MEDEVM frames progressive-evidence diagnosis as a sequential wait-or-answer task and compares submission timing with an annotated boundary for sufficient evidence. Its benchmark is constructed from case reports and evaluated with structured model outputs and submission-related metrics.

Task Formulation

  • Each case presents one atomic observation per turn; WAIT advances the sequence, while ANSWER irreversibly ends the trajectory.The evidence schedule is fixed, so actions determine when a trajectory ends rather than which observation arrives next.Source: §3 · p. 3

Evidence-Value Misalignment

  • The sufficiency boundary is the earliest turn when the gold diagnosis becomes uniquely identifiable; EVM measures mismatch between that boundary and submission timing.A correct guess before the boundary is a premature EVM error, whereas an incorrect answer after it is a diagnostic error rather than a timing mismatch.Source: §3 · p. 3

Benchmark Construction

  • The final MEDEVM benchmark contains 1,050 cases constructed from published case reports, with nine plausible confounders and unanimously annotated sufficiency boundaries.Cases are filtered, checked for diagnostic leakage, segmented into chronological atomic evidence, and jointly verified by a model and medically trained students.Source: §3 · p. 4

Evaluation Protocol

  • Evaluation covers nine LLMs using structured JSON responses and reports diagnosis submission accuracy, premature submission rate, and normalized gold-answer belief.A non-intervening probability probe measures candidate beliefs in a cloned context without altering the original trajectory.Source: §3 · p. 4

4 HOW EVM ARISES: EVIDENCE AND DIAGNOSIS SUBMISSION DIVERGE

Four controlled experiments show that evidence sufficiency, diagnostic confidence, and submission timing can diverge: models mismanage when to answer, depend on evidence order, and remain influenced by misleading observations.

RQ1: Submission timing

  • All nine models exhibit substantial EVM: greater capability changes its form rather than steadily reducing it, while thinking raises early submission to 94.1–99.7% across seven models.Larger and later models reduce early submission but often wait despite sufficient evidence or never answer.Source: §4 · pp. 4–5, Figure 3 · p. 4

RQ2: Diagnostic knowledge

  • At t∗, mean gold belief ranges from 0.70 to 0.86, but submission-point belief varies more; thinking lowers it and increases ECE in all seven tested models.The findings suggest models possess relevant diagnostic knowledge, but knowledge alone does not ensure appropriate submission timing.Source: §4 · p. 5, Figure 4 · p. 5, Table 2 · p. 5

RQ3: Evidence order

  • Reordering the same pre-t∗ evidence changes 29.0–52.9% of final diagnosis outcomes, despite low mean JSD of 0.009–0.044 among pairs reaching t∗ in both orders.The sequences contain identical evidence sets at t∗.Source: §4 · p. 5, Figure 5 · p. 6

RQ4: Misleading evidence

  • Stronger misleading evidence raises belief in an incorrect diagnosis and its submission rate, reducing final accuracy by up to 26.7 percentage points.Its influence persists after all original evidence arrives; misleading evidence inserted after t∗ also reduces how often the correct diagnosis ranks highest.Source: §4 · p. 6, Figure 6 · p. 6

5 WHY EVM MATTERS: PREDICTING AND PREVENTING ERRORS

EVM features add information beyond confidence for predicting submission timing and diagnostic errors, and deferring premature submissions improves accuracy. Natural Gate yields net accuracy gains across nine models.

Predicting submission and errors

  • Adding EVM features improves AUROC for seven of nine LLMs in both tasks, with pooled AUROC rising from 0.835 to 0.854 for next-turn submission and 0.845 to 0.879 for submission errors.Next-turn log loss improves for eight models.Source: §5 · p. 7, Table 3 · p. 7

Preventing premature submission

  • 29.6–48.4 percentage-point accuracy gains across nine models result when Natural Gate defers premature submissions until t∗.Every confidence interval is above zero; 32.9–50.5% of intervened cases change from failure to success, while far fewer change in the opposite direction.Source: §5 · p. 7, Figure 7 · p. 7
  • Natural Gate replaces attempted submissions before t∗ with WAIT, then restores the original action prompt at the sufficiency boundary.It is applied to cases where the model originally answered before t∗.Source: §5 · p. 7, Figure 7 · p. 7

6 EVD-HARNESS: EVIDENCE-VERIFIED DIAGNOSIS HARNESS

EVD-Harness separates diagnosis proposals from submission, using contrastive diagnostic knowledge and evidence review to guide and authorize decisions. It improves accuracy across five models while reducing premature and perturbation-sensitive decisions.

6 EVD-HARNESS: EVIDENCE-VERIFIED DIAGNOSIS HARNESS

  • EVD-Harness decouples diagnosis generation from submission through an offline Wiki and three online modules for evidence management, proposal and witness verification, and submission control.The modules manage context and calls of a frozen LLM, using retrieved knowledge to guide diagnosis and current evidence to authorize submission.Source: §6 · p. 7

6.1 OFFLINE DIAGNOSTIC WIKI CONSTRUCTION

  • The Contrastive Diagnostic Wiki organizes reviewed case-report knowledge into linked disease pages, test pages, and bridges that record confusable diseases, shared presentations, and distinguishing findings.Figure 8 depicts the Wiki guiding diagnosis and authorizing answers grounded in current evidence.Source: §6.1 · p. 7, Figure 8 · p. 8

6.2 ONLINE DIAGNOSIS CONTROL

  • The online control flow maintains the complete evidence record, proposes a diagnosis or WAIT, and separately reviews each proposed diagnosis for support and missing information.The witness supplies evidence citations; citation checking verifies that each cited passage is copied from the patient record, and submission requires witness acceptance plus valid citations.Source: §6.2 · p. 8

6.3 MAIN RESULTS

  • Accuracy improves for all five models, with fewer premature submissions and unanswered cases; diagnosis changes after reordering and incorrect target submissions under strong misleading evidence also decline.The reported gains combine improved accuracy on original cases with more consistent decisions under evidence perturbations.Source: Table 4 · p. 9, §6.3 · p. 9

6.4 ADDITIONAL EVALUATIONS

  • 3.5–9.0 points: removing Evidence Witness while retaining CDW lowers accuracy; removing CDW while retaining the witness lowers accuracy by 9.0–14.5 points.The ablations indicate that retrieved knowledge and evidence review each contribute to diagnostic success.Source: §6.4 · p. 9, Table 5 · p. 9
  • Full CDW retrieval has the highest accuracy for all five models compared with random sampling and BM25 retrieval.Contrastive retrieval uses BM25-selected pages to show shared and distinguishing findings; full CDW follows linked pages over time.Source: §6.4 · p. 9, Table 6 · p. 9

7 CONCLUSION

The paper introduces MEDEVM to evaluate diagnosis submission against evidence sufficiency and identifies evidence-value misalignment across language models. It validates the diagnostic relevance of EVM and reports that EVD-Harness improves accuracy while reducing sensitivity to evidence perturbations.

7 CONCLUSION

  • EVD-Harness improves accuracy by 12.0–51.1 percentage points across five LLMs and reduces sensitivity to evidence perturbations.It retrieves contrastive diagnostic knowledge to guide proposals, then uses independent evidence review and citation checks to authorize submission.Source: §7 · p. 9
  • MEDEVM comprises 1,050 cases and evaluates diagnosis submission against annotated evidence sufficiency; across nine LLMs, the paper identifies EVM as misalignment between submission timing and evidence sufficiency.Evidence order and misleading evidence can redirect decisions even when the correct diagnosis is recognized, and scaling or thinking does not consistently resolve the mismatch.Source: §7 · p. 9
  • EVM provides information about diagnosis submission and correctness beyond model confidence, and preventing premature submission improves accuracy.The conclusion describes EVM as signaling impending errors.Source: §7 · p. 9

LIMITATIONS

The authors identify limits in benchmark representativeness, interaction scope, and sufficiency-boundary review. They also report privacy screening and describe plans to release experiment code.

LIMITATIONS

  • MEDEVM’s published case reports do not represent routine clinical prevalence, and broader sampling across care settings would strengthen external validity.The benchmark also uses fixed candidates and scheduled evidence; active questioning and open-ended diagnosis remain to be studied.Source: LIMITATIONS · p. 10
  • Human review covers only a subset of sufficiency-boundary annotations, so individual boundaries may remain imprecise and warrant broader expert review.Model agreement and sampled audits support annotation quality.Source: LIMITATIONS · p. 10
  • The authors report that published case reports had already been de-identified and that screening checked for potentially identifying information.They state that the study raises no specific ethical concerns beyond data privacy and plan to release experiment code to support reproducibility.Source: ETHICS STATEMENT · p. 10, REPRODUCIBILITY STATEMENT · p. 10

A1 DETAILED CONSTRUCTION OF THE MEDEVM BENCHMARK

The appendix details how MEDEVM cases, evidence sequences, diagnostic options, and sufficiency boundaries were constructed and validated. The final benchmark is selected from quality-controlled dynamic-diagnosis items.

A1.1 CASE COLLECTION AND STRUCTURAL FILTERING

  • Cases are drawn from a 75,434-report pool and structurally filtered for clinical history, diagnostic tests, and evidence spanning presentation, basic workup, and advanced confirmation.After filtering, 36,020 reports proceed to evidence and option construction.Source: A1.1 CASE COLLECTION AND STRUCTURAL… · pp. 12–13

A1.2 EVIDENCE AND OPTION CONSTRUCTION

  • Each case is converted into a ten-way question with a confirmed diagnosis and nine plausible confounders, then screened for a unique answer and sufficient difficulty.Evidence items are segmented into minimal clinical units and ordered by disclosure in the source report; the screening retains 7,472 questions for boundary annotation and validation.Source: A1.2 EVIDENCE AND OPTION CONSTRUCTION · p. 13

A1.3 SUFFICIENCY BOUNDARY AND QUALITY VALIDATION

  • Three models independently mark the earliest evidence position that distinguishes the correct diagnosis from every competing candidate, and only unanimous boundary judgments are retained.Automated checks and an independent validator review the source case, candidate set, and proposed boundary; five medically trained graduate researchers assess 200 sampled items, with 92.5% item-level agreement and 94.0% majority-rated accurate annotation.Source: A1.3 SUFFICIENCY BOUNDARY AND QUALITY… · pp. 13–14
  • Quality control yields 1,810 valid dynamic-diagnosis items, from which stratified sampling by disease system and empirical difficulty selects the final 1,050 cases.The benchmark covers 953 distinct diagnoses across 24 disease systems and contains 26,869 atomic evidence items; each case has ten candidates and averages 25.6 evidence items.Source: A1.3 SUFFICIENCY BOUNDARY AND QUALITY… · p. 14

A2 METRIC DEFINITIONS

The appendix defines measures that separate submission timing from correctness, quantify diagnostic and action beliefs, and evaluate evidence-order effects, prediction, and intervention outcomes.

A2.1 DIAGNOSIS OUTCOMES AND SUBMISSION TIMING

  • Submission accuracy counts only answered cases, whereas diagnostic success treats incorrect submissions and non-answers as errors across all cases.Submission rate measures how often a model produces any diagnosis; submission timing categories distinguish premature, boundary, delayed, and no submission.Source: A2.1 DIAGNOSIS OUTCOMES AND SUBMISSION… · p. 14

A2.2 DIAGNOSTIC BELIEFS, ACTION PROBABILITIES, AND CALIBRATION

  • Candidate probabilities represent favored diagnoses, while action probabilities measure propensity to answer rather than wait at a given evidence checkpoint.Gold belief is probability assigned to the correct diagnosis; ECE measures agreement between confidence and observed correctness.Source: A2.2 DIAGNOSTIC BELIEFS, ACTION… · p. 15

A2.3 EVIDENCE-ORDER AND MISLEADING-EVIDENCE EFFECTS

  • Evidence-order comparisons use JSD for candidate distributions and disagreement rates for submitted diagnoses, with eligible permutations averaged equally by source case.JSD is measured at the sufficiency boundary only when both orders reach it; misleading-evidence effects compare matched strong and weak insertions, with positive values indicating harm.Source: A2.3 EVIDENCE-ORDER AND… · pp. 15–16

A2.4 ERROR PREDICTION AND NATURAL GATE OUTCOMES

  • Prediction metrics assess next-turn submission and errors at first submission, while paired Natural Gate outcomes contrast corrected errors with correct diagnoses lost.Natural Gate's paired accuracy gain includes failures to submit; its full-benchmark gain scales that contrast by the share of cases originally submitted prematurely.Source: A2.4 ERROR PREDICTION AND NATURAL GATE… · p. 16
  • Paired confidence intervals use 2,000 source-case bootstrap resamples, retaining each sampled case's checkpoints and intervention versions together.The 2.5th and 97.5th percentiles define the reported 95% interval.Source: A2.4 ERROR PREDICTION AND NATURAL GATE… · p. 16

A3 EXPERIMENTAL DETAILS AND ADDITIONAL MECHANISM ANALYSES

The appendix specifies the sequential interaction and probability measurements, then tests timing sensitivity, reasoning mode, evidence order, text perturbations, and misleading evidence. Across these analyses, submission timing and diagnoses remain sensitive to evidence presentation and strength.

A3.1 SHARED INTERACTION AND PROBABILITY MEASUREMENT

  • The protocol gives ten candidate diagnoses and one evidence item per turn, requiring the model to wait or submit; first submission ends the trajectory.Belief and action probabilities are scored in cloned contexts at fixed prefixes, without adding probe outputs to the ongoing interaction.Source: A3.1 SHARED INTERACTION AND… · pp. 17–18

A3.2 DIAGNOSIS SUBMISSION TIMING AND BELIEF AT THE SUFFICIENCY BOUNDARY

  • The RQ2 comparison measures gold belief at actual submission and at the sufficiency boundary, using different available case sets because early submissions end trajectories.RQ1 follows each case's original evidence order and reports timing categories separately from correctness.Source: A3.2 DIAGNOSIS SUBMISSION TIMING AND… · p. 18

Sensitivity to boundary tolerance.

  • Allowing one- or two-turn tolerance raises aligned fractions by 1.6–13.4 or 3.3–21.0 percentage points, but substantial timing mismatch or non-response remains.With thinking enabled, premature-submission rates remain above corresponding non-thinking rates even at one- and two-turn tolerance.Source: Sensitivity to boundary tolerance · pp. 18–19, Table A1 · p. 18

A3.3 THINKING-MODE EVALUATION AND BELIEF MEASUREMENT

A3.4 EVIDENCE ORDER: COMPARING FINAL DIAGNOSES AND BOUNDARY BELIEFS

  • Reordering evidence changed 29.0–52.9% of final outcomes; among boundary-reaching pairs, other models' low mean JSD coexisted with 6.2–20.6% diagnosis disagreement.Qwen3-8B contributed only seven boundary-observed source cases because it usually submitted earlier.Source: A3.4 EVIDENCE ORDER: COMPARING FINAL… · p. 20, Table A3 · p. 20

A3.5 REPEATED EVIDENCE, IRRELEVANT TEXT, AND CANDIDATE LABELS

  • Repeated or irrelevant text changed immediate diagnosis outputs in 5.5–34.4% of comparisons, and relabeling candidates changed final outcomes in 9.7–28.8% of cases.The label control preserved diagnosis text, row order, and evidence; the repeated-text comparison measured immediate responses at shared prefixes.Source: A3.5 REPEATED EVIDENCE, IRRELEVANT… · pp. 20–21, Figure A2 · p. 21

A3.6 MISLEADING EVIDENCE: CONSTRUCTION, MEASUREMENTS, AND RESULTS

  • Stronger misleading evidence harmed every model on all five effects, with target belief remaining higher even after original evidence arrived.Effects were positive with confidence intervals excluding zero; the boundary review found validity in 553 cases, from which 318 were selected for intervention experiments. Qwen3-8B showed smaller submission and accuracy changes despite a belief shift.Source: A3.6 MISLEADING EVIDENCE:… · pp. 21, 23, Table A4 · p. 23

A4 ADDITIONAL DIAGNOSTIC-IMPACT ANALYSES

Appendix A4 tests whether evidence-value features predict submission behavior and errors, whether premature submissions miss diagnostic information, and whether delaying submission changes outcomes. Its analyses find predictive value beyond confidence, improved beliefs after sufficient evidence arrives, and higher accuracy under Natural Gate.

A4.1 SEPARATING SUBMISSION PREDICTION FROM ERROR ASSESSMENT

A4.2 DIAGNOSTIC BELIEFS AFTER RECEIVING SUFFICIENT EVIDENCE

  • 49.2–78.4% of incorrect premature submissions had the correct diagnosis as the highest-probability candidate after evidence reached the sufficiency boundary.Across nine LLMs, mean probability assigned to the correct diagnosis increased by 0.245–0.333 after the boundary-reaching evidence item.Source: A4.2 DIAGNOSTIC BELIEFS AFTER… · p. 27
  • For cases submitted exactly one turn before sufficiency, the correct top-probability diagnosis at the boundary exceeded original accuracy by 19.7–33.3 percentage points.The original diagnosis ends the trajectory just before the evidence item that can resolve the case arrives.Source: A4.2 DIAGNOSTIC BELIEFS AFTER… · p. 27

A4.3 NATURAL GATE: DELAYING SUBMISSION UNTIL EVIDENCE IS SUFFICIENT

  • 32.9–50.5% of selected cases changed from an incorrect diagnosis to a correct one under Natural Gate, while 1.9–6.2% lost an originally correct diagnosis.Gated accuracy exceeded original accuracy for every LLM.Source: A4.3 NATURAL GATE: DELAYING SUBMISSION… · pp. 27–28, Table A8 · p. 28
  • 29.6–48.4 percentage-point net accuracy gains followed Natural Gate intervention across originally premature submissions, with all nine paired confidence intervals above zero.Gains remained positive after accounting for newly incorrect diagnoses and cases ending without a submission.Source: A4.3 NATURAL GATE: DELAYING SUBMISSION… · p. 28, Table A8 · p. 28
  • Benchmark-wide gains depend on both the prevalence of premature submissions and the intervention’s effect on those cases.Qwen3-8B’s 45.2-point selected-case gain corresponded to 44.9 points across the benchmark, whereas Kimi-K2.5’s 33.7-point selected-case gain corresponded to 5.3 points.Source: A4.3 NATURAL GATE: DELAYING SUBMISSION… · p. 28

A5 ADDITIONAL EVALUATION OF EVD-HARNESS

Appendix A5 evaluates EVD-HARNESS on dynamic diagnosis, component and execution controls, evidence perturbations, and static medical question answering. The reported results show better diagnosis outcomes and timing, with robustness and transfer benefits across the tested settings.

A5.1 EVALUATION PROTOCOL AND DIAGNOSIS OUTCOMES

A5.3 SEPARATING KNOWLEDGE, VERIFICATION, AND CONTEXT EFFECTS

  • Removing Evidence Witness lowered accuracy by 3.5–9.0 percentage points, while removing CDW with the witness retained lowered it by 9.0–14.5 points.Witness removal also increased premature incorrect submissions; CDW supports candidate distinction, while the witness assesses whether a proposal is supported for submission.Source: A5.3 SEPARATING KNOWLEDGE,… · pp. 30–31, Table A12 · p. 30, Table A13 · p. 30
  • Contrastive organization of the same BM25-retrieved pages raised accuracy by 4.0–6.0 percentage points and reduced premature incorrect submissions for every model.Full CDW achieved the highest diagnostic accuracy across the five models.Source: A5.3 SEPARATING KNOWLEDGE,… · p. 31, Table A14 · p. 31
  • Shortening visible action history was associated with 4.0–9.0-point accuracy increases in the observed results, and the Direct condition showed that execution procedure also affects outcomes.The component and retrieval comparisons held history window and execution protocol fixed.Source: A5.3 SEPARATING KNOWLEDGE,… · pp. 31–32, Table A15 · p. 31

A5.4 SENSITIVITY TO EVIDENCE PERTURBATIONS

  • Answer-flip rates under repeated, irrelevant, or paraphrased evidence fell from 8.3–31.7% with LLM-only to 1.1–3.8% with EVD-HARNESS.Across the three insertion conditions, reductions averaged 74%–96%.Source: A5.4 SENSITIVITY TO EVIDENCE… · p. 32, Table A16 · p. 32
  • EVD-HARNESS reduced premature submission of the designated incorrect diagnosis under high-strength misleading evidence for all five models.The analysis used 318 paired cases with corrective evidence retained in the underlying case.Source: A5.4 SENSITIVITY TO EVIDENCE… · p. 32

A5.5 TRANSFER TO STATIC MEDICAL QUESTION ANSWERING

Related papers

Loading 2609.35627v1…