PaperlayerView exact revision on arXiv

Source-linked AI summary

Grounding Healthcare LLMs in a Causal Knowledge Graph: Framework, Metrics, and a Cardiovascular Pilot

Ummara Mumtaz, Aimen Noor, Awais Ahmed

arXiv:2608.15382v1cs.AIcs.CLcs.IRq-bio.QM

TL;DR

Healthcare LLM evaluations largely emphasize answer accuracy rather than intervention-oriented reasoning about mechanisms, harms, evidence, and uncertainty. This paper introduces a graph-centered, multidimensional framework and cardiovascular pilot showing that grounding conditions can be distinguished beyond single-answer accuracy.

  • Problem

    Existing healthcare LLM evaluations predominantly use multiple-choice accuracy and inadequately assess mechanisms, harms, contraindications, verifiable evidence, and unsupported claims.

  • Method

    The framework combines a provenance-preserving causal graph, scenario-conditioned subgraph extraction, controlled grounding conditions, assertion-linked scoring, and category-balanced scenarios spanning eight failure modes.

  • Results

    The cardiovascular pilot showed that multidimensional metrics distinguished grounding conditions, with C4 achieving causal edge F1 0.838 while revealing patterns obscured by single accuracy.

  • Takeaways & Limitations

    Healthcare LLM benchmarks should treat answer accuracy as one axis alongside unsupported claims, evidence alignment, harms, contraindications, and uncertainty.

  • Takeaways & Limitations

    The empirical evaluation is a single-model cardiovascular pilot and does not establish clinical performance, external validity, or superiority of any grounding condition.

Abstract

from arXiv · show

Large language models (LLMs) are increasingly proposed for healthcare decision support, but their evaluations still reward single-answer accuracy rather than reasoning about interventions, mechanisms, harms, evidence, and uncertainty. We propose a reproducible, graph-centered evaluation framework for intervention-oriented LLM behavior in healthcare and stress-test it in a cardiovascular pilot. The framework has four components: (i) a domain causal knowledge graph in which assertions are first-class, provenance-preserving nodes with stable identifiers; (ii) a scenario-conditioned subgraph extraction step that, given any clinical scenario, retrieves the relevant reified-assertion subgraph; (iii) four controlled grounding conditions that vary how the retrieved subgraph is composed into the model's context (ungrounded C1, knowledge-graph C2, causal-graph C3, integrated C4); and (iv) an automated scoring pipeline, anchored on assertion identifiers, that computes intervention accuracy, and other evaluation measures on a single pass. To test the framework, we built a category-balanced scenario generator across eight reasoning failure modes and instantiated it on a cardiovascular graph. The metric panel discriminates conditions along interpretable, non-redundant axes: C4 obtains the strongest causal edge F1 (0.838), adverse-effect F1 (0.833), evidence accuracy (0.738), and unsupported claim rate (0.114), while C1 obtains the highest raw intervention accuracy (0.948) with no measurable causal or evidential grounding.

1. Introduction

The paper introduces a causal, provenance-preserving framework for evaluating intervention-oriented healthcare LLM behavior beyond single-answer accuracy. It instantiates the framework in a cardiovascular intervention-planning pilot using scenario-conditioned graph grounding and multidimensional automated scoring.

  • Motivation: Existing clinical LLM evaluations rely predominantly on multiple-choice benchmarks that assess whether models select the correct response.These benchmarks support standardized comparison but capture only a limited dimension of clinical performance.
  • Motivation: The framework evaluates intervention appropriateness alongside causal-mechanism plausibility, supporting evidence, potential harms, and uncertainty.It reflects the shift from predictive modeling toward causal decision support.
  • Framework: Its causal and evidence graph represents each biomedical claim as a reified assertion node with a stable assertion_id and preserved provenance.Scenario-conditioned Cypher retrieval returns the relevant reified-assertion subgraph for each clinical scenario.
  • Framework: Four controlled grounding conditions—ungrounded C1, knowledge-graph C2, causal-graph C3, and integrated causal-knowledge-graph C4—vary retrieved context while holding scenarios fixed.The scoring pipeline measures intervention accuracy, causal edge precision/recall/F1, adverse-effect F1, contraindication recall, evidence accuracy, unsupported claim rate, and uncertainty correctness.
  • Evaluation setting: The cardiovascular pilot uses graph-anchored scenarios containing patient context, a decision target, candidate actions, and a gold answer with the preferred intervention and causal rationale.Example scenarios include resistant hypertension with impaired renal function and pregnancy in which ACE inhibitors are contraindicated.

2. Related Work

Prior medical LLM benchmarks emphasize single-answer correctness, while biomedical graph methods mainly improve retrieval and downstream tasks without evaluating grounded reasoning at the assertion level. This framework addresses that gap by using addressable, provenance-preserving assertions to score causal and evidential claims reproducibly.

  • Benchmark limitations: Medical LLM benchmarks primarily evaluate final-answer accuracy, collapsing reasoning quality into a single scalar.Examples include MedQA, MedMCQA, PubMedQA, MMLU medical subsets, MultiMedQA, HealthBench, and clinical HELM extensions.
  • Benchmark limitations: Biomedical knowledge graphs and graph-aware retrieval methods provide structured context and improve downstream question answering and summarisation, but evaluation infrastructure remains comparatively limited.Existing benchmarks generally measure whether the final answer changed rather than grounded reasoning quality.
  • Assertion-level representation: Nanopublications, Biolink, and W3C PROV established scientific assertions as addressable objects with typed identity, qualifiers, evidence, and provenance.These traditions provide the conceptual and infrastructural basis for representing claims independently of papers or unqualified triples.
  • Assertion-level evaluation: Existing graph-based benchmarks usually retrieve biomedical context but score outputs as final-answer strings or free-text rationales, severing evaluation from identifiable graph assertions.This forces surface-form matching or manual alignment of claims to unlabeled edges.
  • Causal and evidential grounding: The framework extends factuality and attribution evaluation to causal mechanism claims by defining unsupported claims through absence of matching subject–predicate–object patterns in the retrieved reified-assertion subgraph.Unsupported-claim rate is computed automatically in the same scoring pass as causal-edge precision and recall.

3. The Framework

The framework evaluates whether healthcare LLMs produce defensible intervention recommendations by linking a provenance-preserving causal knowledge graph to scenario-specific retrieval, controlled grounding, and assertion-anchored scoring. It supports interpretable comparisons through balanced scenarios and repeated evaluation under four context conditions.

  • Framework components: The framework comprises a reified-assertion knowledge graph, scenario-conditioned subgraph extraction, four controlled grounding conditions, and automated scoring anchored to assertion_ids.These components connect cited knowledge, scenario-relevant retrieval, fixed context organization, and comparison of model outputs against the same reference graph.
  • Knowledge graph: Each biomedical claim is stored as a first-class node with a stable identifier, provenance, confidence, validation state, and context, serving as the evaluation’s single source of truth.Reification also allows evidence links to attach to specific claims rather than opaque entity-to-entity edges.
  • Scenario-conditioned extraction: Parameterized Neo4j queries retrieve only the drugs, mechanisms, outcomes, contraindications, adverse effects, and evidence sources relevant to each clinical scenario.The extracted subgraph, rather than the full graph, is composed into the model context so scenario relevance is enforced during retrieval.
  • Controlled grounding conditions: C1 is ungrounded, C2 provides a knowledge-graph view, C3 provides a causal-graph view, and C4 integrates both while varying context type rather than volume.The same scenario is run under all four conditions, isolating context organization as the intended source of behavioral differences.
  • Automated scoring: The scoring pass compares parsed outputs with gold assertion_id sets to compute intervention accuracy, causal-edge precision/recall/F1, adverse-effect F1, contraindication recall, evidence accuracy, unsupported-claim rate, and uncertainty correctness.Using the same graph, identifiers, and scoring pass makes the metric panel coherent and exposes errors such as missed mediators, missed adverse effects, and fabricated claims.
  • Evaluation instruments: Scenarios are evenly balanced across eight causal-reasoning failure modes, and each scenario is presented under all four conditions for within-scenario comparisons.The repeated-measures protocol yields 320 runs for one model across 80 scenarios, while the scenario generator defines clinical context, decision target, candidate actions, and gold intervention, causal, and evidential structures.

4. Results

The cardiovascular pilot shows that the metric panel separates grounding conditions and scenario categories along distinct, interpretable axes rather than reducing intervention reasoning to raw accuracy alone. C4 leads on causal, adverse-effect, evidential, and unsupported-claim metrics, while diagnostics expose benchmark and output-interpretation effects.

  • Overall metric panel: C4 achieved the highest causal edge F1 (0.838), adverse-effect F1 (0.833), and evidence accuracy (0.738), plus the lowest unsupported claim rate (0.114).C1 instead achieved the highest raw intervention accuracy (0.948), while grounded conditions scored 0.873 for C2, 0.808 for C3, and 0.886 for C4.
  • Scenario categories: C4 showed perfect intervention accuracy and causal recall in multiple-pathway scenarios, with evidence accuracy of 1.000 and unsupported claim rate of 0.000.In adverse-effect pathway scenarios, C4 achieved intervention accuracy, causal precision, causal recall, and adverse-effect F1 of 1.000.
  • Scenario categories: All grounded conditions scored 0.000 intervention accuracy but 1.000 uncertainty correctness on incomplete or uncertain evidence scenarios.This dissociation separates recognition of evidential insufficiency from producing the gold intervention decision.
  • Paired within-scenario deltas: Relative to C2, C4 improved intervention accuracy by +0.013, causal precision by +0.284, causal recall by +0.101, causal F1 by +0.259, adverse-effect F1 by +0.100, and evidence accuracy by +0.033.C4 also reduced unsupported claim rate by −0.145, while contraindication recall was 0.200 lower.
  • Diagnostics: C1’s high intervention accuracy was partly attributable to incomplete or uncertain evidence scenarios, where grounded conditions scored 0.000 intervention accuracy but 1.000 uncertainty correctness.The result shows why uncertainty correctness must be measured separately from a single accuracy endpoint.
  • Diagnostics: C4’s lower contraindication recall arose in two scenarios where the model recommended amlodipine but returned an empty contraindication field despite pregnancy being the expected gold context.The relevant contraindication information was present, suggesting a narrow model interpretation versus a broader scorer interpretation of the output field.

5. Discussion

The discussion frames the work as a reproducible, multidimensional evaluation framework for causal, evidential, harm-related, and uncertainty-aware healthcare LLM behavior, not a clinical-performance claim. Its single-model cardiovascular pilot demonstrates implementability and discrimination, while broader model and domain validation remains future work.

  • Framework contribution: The manuscript’s principal contribution is an evaluation framework combining provenance-preserving causal graphs, scenario-conditioned extraction, controlled grounding conditions, and assertion-anchored automatic scoring.These components define how claims move from the knowledge graph into model evaluation.
  • Limitations: The single-model cardiovascular pilot demonstrates that the framework is implementable and discriminative, but does not establish clinical superiority for any grounding condition.Comparative model claims and effect-size estimates are deferred to a planned multi-model repeated-measures phase.
  • Implications: Healthcare LLM benchmarks should treat final-answer accuracy as one axis alongside unsupported claims, evidence alignment, harms, contraindications, and uncertainty.The framework measures these dimensions explicitly rather than relying on single-answer agreement alone.
  • Implications: Structured biomedical graphs can serve as experimental instruments whose contents and organization are varied to test responses to different supplied structures.This extends their role beyond retrieval resources.
  • Future work: Future work will evaluate broader frontier and openweight LLMs with repeated measures and test framework transfer beyond cardiovascular care.These studies will examine whether the category structure and metric panel generalize across models and domains.

6. Conclusion

The paper proposes a graph-centered framework for evaluating intervention-oriented healthcare LLM behavior through provenance-preserving causal knowledge, scenario-conditioned retrieval, controlled grounding, and multidimensional scoring. A 313-run cardiovascular pilot with GPT-5.4 demonstrates end-to-end implementation and shows that multidimensional metrics distinguish grounding and failure patterns obscured by single-answer accuracy.

  • Framework: The framework combines a provenance-preserving causal knowledge graph, scenario-conditioned subgraph extraction, controlled grounding conditions, and assertion-linked multidimensional scoring.Assertions are reified and linked to stable assertion_ids.
  • Framework: A category-balanced scenario generator produces structured evaluations of intervention-oriented healthcare LLM behavior.
  • Cardiovascular pilot: 313 scored runs with GPT-5.4 demonstrate that the cardiovascular pilot implements the framework end to end.
  • Cardiovascular pilot: The evaluation metrics distinguish performance across grounding conditions and scenario categories.
  • Cardiovascular pilot: Multidimensional evaluation reveals output-schema misalignment separately from substantive knowledge gaps, patterns obscured by a single accuracy measure.

Statements and Declarations · Funding

The manuscript reports that no funding was received for its preparation.

  • Funding: No funding was received to assist with preparing the manuscript.

Competing interests

The authors report no relevant financial or non-financial competing interests.

  • The authors have no relevant financial or non-financial interests to disclose.

Ethics approval

The study used only synthetic clinical scenarios generated from a curated biomedical knowledge graph and involved no human participants, data, or biological material; institutional review board approval was therefore not required.

  • Ethics approval: No human participants, human data, or human biological material were involved, so institutional review board approval was not required.The benchmark used synthetic clinical scenarios and automated scoring metrics without collecting or analyzing patient-level data.

Consent to participate and consent to publish

Consent to participate and consent to publish were not applicable because the work did not include data from identifiable individuals.

  • Consent was not applicable because the study used no data from identifiable individuals.

Declaration on Generative AI

The authors used an AI assistant for copy editing the manuscript and retained responsibility for its content.

  • An AI assistant supported copy editing for readability, grammar, and formatting; the authors reviewed and edited the content as needed and accept full responsibility.
Loading 2608.15382v1…