PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenMTB-Audit: Exposing Over-Refusal and Clinical Expert Perspectives in LLM-Based Molecular Tumor Board Safety Evaluation

Negin Ashrafi, Jia Luo, Stacey M. Frumm, Roxana Daneshjou

arXiv:2610.01497v1cs.AIcs.LG

TL;DR

This paper addresses the challenge of distinguishing unsupported therapies from evidence-supported recommendations that still require clinical review. It introduces a synthetic NSCLC benchmark and a deterministic audit framework; LLMs often lose the Partially Supported label, while the framework reports lower over-refusal and higher accuracy. The expert annotation study found disagreement concentrated at the boundary between information sufficiency and treatment optimization.

  • Problem

    Prior work gave less attention to evidence-supported therapies requiring clinical review, although clinical context can determine whether their implementation is appropriate.

    Source: §1 · p. 2

  • Method

    The authors introduce OpenMTB-Audit, a 500-case synthetic NSCLC benchmark, and MTB-AuditAgent, a deterministic framework separating evidence verification, information assessment, classification, and abstention.

    Source: §1 · p. 2

  • Results

    Across eight LLM configurations, 83.3–100% of true Partially Supported cases lost that label; MTB-AuditAgent had 6.7% over-refusal and 91.2% accuracy (95% CI: 88.6–93.6%).

    Source: §1 · p. 2, §11 · p. 13

  • Takeaways & Limitations

    Separating evidence validity, information sufficiency, and review need preserves clinically meaningful distinctions in recommendation audits.

    Source: §10 · p. 12, §11 · p. 13

    The benchmark covers only NSCLC and 15 molecular profiles, and expert-subset refinement means reported agreement is post-refinement rather than prospective external validation.

    Source: §10 · p. 12

Abstract

from arXiv · show

Molecular tumor boards integrate genomic findings, clinical context, and therapeutic evidence to support precision oncology. As AI enters this workflow, a key safety challenge is distinguishing truly unsupported recommendations from evidence-supported options that still require oncologist review because of incomplete information, poor ECOG performance status, or other clinical caveats. We introduce OpenMTB-Audit, an open-source benchmark of 500 synthetic non-small cell lung cancer cases spanning five adversarial error categories and four safety labels: Supported, Partially Supported, Unsupported, and Insufficient Information. Across eight large language model configurations, we identify pervasive over-refusal: all LLM configurations failed to retain the Partially Supported label in 83.3-100% of true Partially Supported cases, achieving high aggregate safety scores through label collapse rather than clinically calibrated reasoning. To address this limitation, we developed MTB-AuditAgent, a deterministic seven-module framework separating evidence verification, missing-information detection, safety classification, and abstention. It reduces over-refusal to 6.7% and achieves 91.2% accuracy (95% CI: 88.6-93.6%). A two-oncologist annotation study found disagreement concentrated at the boundary between information sufficiency and treatment optimization, underscoring the need to preserve clinically meaningful distinctions.

1 Introduction

Molecular tumor boards must assess therapies against genomic evidence and patient context, while AI safety evaluation has paid less attention to evidence-supported recommendations that still require clinical review. This paper introduces a benchmark and audit framework to study whether systems preserve that intermediate judgment.

1 Introduction

  • Prior evaluations emphasized recommendation correctness, evidence retrieval, or clearly unsafe outputs, with less attention to evidence-supported therapies requiring clinical review.Patient-specific factors such as performance status, incomplete documentation, prior treatment, and sequencing can affect whether an otherwise appropriate therapy should be implemented.Source: §1 · p. 2
  • The paper identifies clinical over-refusal as failure to preserve the Partially Supported label, instead classifying review-worthy therapies as Unsupported or Supported.This differs from a textual refusal: the model responds but loses the calibrated intermediate label.Source: §1 · p. 2
  • OpenMTB-Audit is an open-source benchmark of 500 synthetic NSCLC cases spanning five adversarial error categories and four clinically informed safety labels.Its labels distinguish evidentiary support from whether available clinical information is sufficient for implementation.Source: §1 · p. 2
  • The authors report that aggregate safety scores can remain high while systems systematically misclassify evidence-supported recommendations that require clinical review.Source: §1 · p. 2
  • MTB-AuditAgent is a deterministic seven-module pipeline combining evidence verification, missing-information detection, four-label classification, and human-referral abstention.Source: §1 · p. 2
  • Expert concordance and qualitative disagreement analyses identify information sufficiency and treatment optimization as key clinical decision boundaries.Source: §1 · p. 2

2 Related Work

Prior work has examined LLMs for molecular tumor board support, clinical evaluation, and safety alignment, but this paper focuses on calibration of a clinically meaningful intermediate label in recommendation auditing.

2.1 LLMs for Molecular Tumor Board Support

  • Prior molecular tumor board systems generate or retrieve treatment options, whereas this work audits existing recommendations and evaluates their label calibration.Related systems include expert-comparison, retrieval-augmented, and multimodal or longitudinal approaches.Source: §2.1 · p. 2

2.2 Clinical LLM Evaluation and Safety

  • Medical knowledge benchmarks and licensing-style exams do not capture workflow safety requirements, and clinical LLM evaluations show task-level variation and persistent errors.Source: §2.2 · p. 3

2.3 Over-Refusal and Safety Calibration

  • Healthcare safety research has documented both harmful compliance and over-refusal on safe medical prompts, including in dual-use health settings.Source: §2.3 · p. 3
  • Here, clinical over-refusal means failing to assign Partially Supported to a recommendation warranting qualified review, whether it is labeled Unsupported or Supported.The authors state that prior work had not quantified this as a label-calibration error in molecular tumor board auditing.Source: §2.3 · p. 3

3 The OpenMTB-Audit Benchmark

OpenMTB-Audit is a synthetic NSCLC benchmark built around the distinction between evidentiary support and the sufficiency of clinical information for assessing a proposed therapy.

3.1 Design Rationale and Data Sources

  • The benchmark focuses on NSCLC, with 15 molecular profiles and 20 FDA-labeled therapies supported by regulatory indications, guidelines, and public evidence resources.Source: §3.1 · p. 3

3.2 Case Construction

  • OpenMTB-Audit contains 500 cases, including clean cases and adversarial cases distributed across error categories that target distinct failure modes.The table caption specifies 60 cases per adversarial category, and Figure 1 provides worked examples of the four reference labels.Source: §3.2 · pp. 3–4, Table 1 · p. 4, Figure 1 · p. 4

3.3 Four-Label Safety Framework

  • Supported cases have adequate evidence and clinical information, while Partially Supported cases have a rationale but require patient-specific oncologist review.Poor ECOG performance status and extensive prior treatment are examples of review-requiring considerations.Source: §3.3 · p. 4
  • Unsupported denotes inadequate molecular or disease-context support, whereas Insufficient Information means missing critical fields prevent a reliable appropriateness determination.The framework treats Partially Supported as valid rationale requiring review, and Insufficient Information as warranting abstention rather than a definitive label.Source: §3.3 · p. 5

4 MTB-AuditAgent

MTB-AuditAgent audits proposed molecular tumor board recommendations through a deterministic workflow that separates evidence checks, information sufficiency, safety classification, and referral for human review.

4 MTB-AuditAgent

  • MTB-AuditAgent audits rather than generates recommendations, producing a structured report after checking support, missing information, and need for human review.Its decision logic is rule-based after evidence retrieval, making predictions transparent and reproducible.Source: §4 · p. 5, Figure 2 · p. 6
  • The Case Parser extracts cancer and treatment context, including alteration, therapy, ECOG status, prior treatment, PD-L1, and brain metastasis status.It also identifies potential variants of uncertain significance and non-actionable alterations.Source: §4 · p. 5
  • The Evidence Retriever matches biomarker, disease context, and therapy to FDA approvals and CIViC records, and the Evidence Verifier checks concordance.The verifier supplies the principal evidence signal to downstream safety classification.Source: §4 · p. 5
  • The Missing Information Detector treats stage, ECOG status, prior treatment, and biomarker validation as critical, while PD-L1 and brain metastasis are contextual.The Safety Classifier combines evidence and information completeness to assign one of four labels.Source: §4 · p. 5
  • The Abstention Module flags cases for oncologist review, and the report generator records the label, abstention recommendation, missing information, evidence, and rationale.Source: §4 · p. 5

5 Experimental Setup

The evaluation compared MTB-AuditAgent with eight LLM-based baselines on all 500 benchmark cases. The baselines used progressively stronger prompting strategies, while MTB-AuditAgent used a deterministic pipeline without LLM inference.

5 Experimental Setup

  • MTB-AuditAgent was evaluated alongside eight LLM-based baselines on all 500 benchmark cases.The agent uses seven deterministic modules and makes no LLM calls at inference time.Source: §5 · p. 6
  • LLM experiments used GPT-4o and GPT-4o-mini via the OpenAI API with temperature 0 and a 512-token maximum.Experiments ran from June to July 2026; prompts, evaluation code, and benchmark data were made available in a public repository.Source: §5 · p. 6
  • The LLM baselines progressed from zero-shot prompting through retrieval, explicit evidence verification, and a structured four-label checklist.The strategies included Base LLM, Simple RAG, RAG+EV, and Prompt-Checklist, each using the specified GPT-4o or GPT-4o-mini configurations.Source: §5 · p. 6

6 Evaluation Metrics

The evaluation defines a weighted Safety Score from 0 to 100 to reflect differences in the clinical impact of errors. It also reports standard classification metrics and tests the stability and significance of comparisons.

6 Evaluation Metrics

  • False approval of unsupported therapies receives the largest Safety Score weight, 0.30, reflecting its risk of harm through active endorsement.Abstention and missing-information detection each receive weight 0.20; adversarial catch rates are weighted according to clinical severity.Source: §6 · pp. 6–7
  • The Safety Score combines six components: Rmiss, abstention F1, missing-information detection, and catch rates for three adversarial error types.Rmiss is the fraction of unsupported therapies incorrectly labeled Supported; the catch rates cover Wrong Drug, Hallucinated Variant, and Unsafe Overconfidence errors.Source: §6 · pp. 6–7
  • The authors report accuracy, macro F1, and class-specific precision and recall alongside the Safety Score.A ±0.05 weight sensitivity analysis preserved system rankings; accuracy confidence intervals used 10,000 bootstrap resamples, and paired comparisons used McNemar’s test.Source: §6 · p. 7

7 Over-Refusal: A Failure of Clinical Safety Calibration

The section defines clinical over-refusal as failure to preserve the Partially Supported label for evidence-supported therapies needing clinical review. It shows that aggregate safety scores can conceal this calibration failure and its class-specific consequences.

7.1 Definition and Quantification

  • Over-refusal is the proportion of ground-truth Partially Supported cases not predicted as Partially Supported, including misclassification as Supported or Unsupported.In this benchmark, the 60 cases involve molecularly appropriate therapies requiring oncologist review, distinguishing treatment-implementation uncertainty from lack of evidence.Source: §7.1 · p. 7, Table 2 · p. 8
  • All LLM configurations almost never assigned the Partially Supported label correctly despite explicit instructions and its operational definition.For GPT-4o configurations, the reported failure rates of 98.3% and 100% were not statistically distinguishable (59/60 vs. 60/60; two-sided Fisher’s exact p = 1.00).Source: §7.1 · p. 7

7.2 Clinical Consequences

  • Safety Scores of 90.1 for Prompt-Checklist GPT-4o and 92.4 for MTB-AuditAgent were close despite substantially different clinical error profiles.The baseline correctly identified 98% of Unsupported cases but failed on all 60 Partially Supported cases.Source: §7.2 · p. 7, Table 3 · p. 8
  • MTB-AuditAgent achieves 0.93 Partially Supported recall, while Prompt-Checklist GPT-4o assigns none of the 60 such cases correctly.Prompt-Checklist GPT-4o classified 31 as Supported and 29 as Unsupported; its Unsupported precision was 0.75, compared with 0.91 for MTB-AuditAgent.Source: §7.2 · pp. 7–8, Table 3 · p. 8

8 Expert Annotation Study

The expert annotation study used a stratified subset of 50 cases to assess benchmark-label plausibility and characterize clinical disagreement. Disagreement clustered around information sufficiency and treatment optimization, while the paper emphasizes clinician review when AI and expert assessments diverge.

8.1 Study Design

  • Two blinded oncologists independently labeled a stratified 50-case subset enriched for difficult examples, and their disagreements were retained rather than adjudicated.Thirty cases were selected from those showing disagreement among evaluated AI systems; the subset was reviewed during MTB-AuditAgent development, so agent agreement is post-refinement concordance rather than held-out validation.Source: §8.1 · pp. 8–9

8.2 Agreement and Sources of Disagreement

  • Oncologist A agreed with all 50 benchmark labels, while Oncologist B agreed on 34 cases (κ = 0.553).The reported inter-oncologist agreement was identical to B-versus-benchmark agreement by construction; the paper characterizes it as moderate.Source: §8.2 · p. 9
  • Fifteen of 16 oncologist disagreements concerned information sufficiency or treatment optimization, with three recurring patterns accounting for 12 cases.The patterns involved missing clinical information (7 cases), ROS1 treatment optimization (3), and ECOG 3 tolerability versus required review (2).Source: §8.2 · pp. 9–10

8.3 Implications for AI Safety

  • On the 50 expert-reviewed cases, MTB-AuditAgent achieved post-refinement κ = 0.833 versus κ = 0.451 for Prompt-Checklist GPT-4o against the benchmark.MTB-AuditAgent correctly classified all ten Partially Supported cases in the subset; the baseline classified all ten as Unsupported.Source: Table 4 · p. 9, §8.3 · p. 10
  • When AI and expert assessments diverge, the paper directs clinicians to consult the M7 audit report and not act on flagged cases without attending-oncologist review.The authors identify distinguishing therapies that need clinical review from those lacking evidentiary support as the primary challenge.Source: §8.3 · p. 10

9 Full Evaluation Results

MTB-AuditAgent outperformed prompting-based systems on the four-label benchmark, while ablations and controlled tests examined which modules and evidence-availability conditions shaped its safety and accuracy.

9.1 Main Results

  • 91.2% accuracy and a Safety Score of 92.4 for MTB-AuditAgent exceeded RAG+EV with GPT-4o’s 55.4% accuracy and Safety Score of 90.9.The accuracy difference was 35.8 percentage points; confidence intervals did not overlap, and a paired McNemar test found a significant case-level difference (p < 0.001).Source: §9.1 · pp. 10–11
  • Structured LLM configurations had Safety Scores of 83.9–90.9 but accuracies of only 46.2–74.0%, whereas MTB-AuditAgent preserved the Partially Supported category.GPT-4o outperformed GPT-4o-mini across matched strategies, but margins were small for unstructured Base LLM and Simple RAG configurations.Source: §9.1 · p. 11

9.2 Module Ablation

  • Removing the Evidence Verifier caused the largest safety drop (−55.0 points), while removing M4 reduced Macro F1 from 0.898 to 0.638 and collapsed the Insufficient Information class.The ablations indicate complementary module functions rather than interchangeable improvements.Source: §9.2 · p. 11, Table 6 · p. 11
  • Removing the ECOG rule reduced Safety Score by 4.6 points; removing M6 left accuracy and Macro F1 unchanged but reduced Safety Score by 13.7 points.M4 assigns Insufficient Information directly; M6 abstention provides a redundant signal for borderline cases rather than the primary means of detecting missing critical information.Source: §9.2 · p. 11

9.3 Benchmark Validity

  • Withheld-profile tests retained a Safety Score of 91.7 across 165 affected cases and detected all predefined violations, while accuracy fell to 47.9% as supported recommendations became Insufficient Information.In a text-only setting, accuracy was 92.2% and Safety Score was 94.5; the authors describe abstention when evidence is unavailable as the intended failure mode.Source: §9.3 · pp. 11–12
  • Figure 4 reports correct-label rates for five error types with N = 60 each and identifies Unsafe Overconfidence as the largest gap.The caption does not provide the category-specific rates.Source: Figure 4 · p. 12

10 Discussion

The discussion argues that aggregate safety scores can conceal label-calibration failures and that expert disagreement clusters around information sufficiency and treatment decisions. It also identifies benchmark, validation, and model-coverage boundaries.

10 Discussion

  • Structured LLMs scored 84–91 on Safety Score yet systematically failed to preserve Partially Supported, while expert disagreement concentrated on missing information, treatment optimization, and performance status.On clearly incorrect recommendations, oncologists agreed almost unanimously; MTB-AuditAgent separates evidence validity, information sufficiency, and review need as decision states.Source: §10 · p. 12
  • OpenMTB-Audit covers only NSCLC and 15 molecular profiles, and its expert subset informed agent refinement, so reported agreement is not prospective external validation.The authors also caution that shared design provenance means benchmark performance is not independent clinical validation.Source: §10 · p. 12

11 Conclusion

The conclusion presents OpenMTB-Audit and MTB-AuditAgent for NSCLC safety evaluation, reporting frequent LLM loss of the Partially Supported label and better-calibrated agent performance. Expert disagreement centered on information sufficiency and treatment optimization.

11 Conclusion

  • Across LLM configurations, 83.3–100% of true Partially Supported cases lost that label, while expert disagreement clustered at information sufficiency and treatment optimization.The two-oncologist annotation study had moderate inter-annotator agreement (κ = 0.553); Oncologist A fully agreed with benchmark labels (κ = 1.000).Source: §11 · p. 13
  • MTB-AuditAgent achieved 91.2% accuracy, a Safety Score of 92.4, 6.7% over-refusal, and 91% precision on Unsupported by separating verification, missing-information assessment, risk routing, and abstention.Its improvement over the best-safety LLM baseline was statistically significant (McNemar p < 0.001), with non-overlapping bootstrap confidence intervals.Source: §11 · p. 13
Loading 2610.01497v1…