PaperlayerView exact revision on arXiv

Source-linked AI summary

KGARevion: An AI Agent for Knowledge-Intensive Biomedical QA

Xiaorui Su, Yibo Wang, Shanghua Gao, Xiaolong Liu, Valentina Giunchiglia, Djork-Arné Clevert, Marinka Zitnik

arXiv:2410.04660v2cs.AI

TL;DR

Medical QA systems must handle specialized biomedical knowledge and the limitations of retrieval and verification. KGAREVION generates knowledge-graph triplets from questions and checks them before answering. The paper reports higher accuracy across evaluated datasets and zero-shot performance on AfriMed-QA.

  • Problem

    Medical LLMs can give inaccurate or incomplete answers to complex queries, while RAG answers depend on retrieved-content quality and often lack post-retrieval verification.

    Source: §1 · pp. 1–2

  • Method

    KGAREVION generates question-relevant triplets with an LLM, verifies them against a knowledge graph, revises errors, and answers from verified triplets.

    Source: §3 · p. 3

  • Results

    6.75% improvement over 15 baseline models across seven datasets is reported, alongside zero-shot generalization on AfriMed-QA.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The evaluations report improved accuracy across multiple-choice and open-ended tasks, including newly curated challenging benchmarks.

    Source: §5 · p. 10

    The authors caution that medical LLMs are not a substitute for professional medical advice and that deployment should be approached cautiously.

    Source: ETHICS STATEMENT · p. 11

Abstract

from arXiv · show

Biomedical reasoning integrates structured, codified knowledge with tacit, experience-driven insights. Depending on the context, quantity, and nature of available evidence, researchers and clinicians use diverse strategies, including rule-based, prototype-based, and case-based reasoning. Effective medical AI models must handle this complexity while ensuring reliability and adaptability. We introduce KGARevion, a knowledge graph-based agent that answers knowledge-intensive questions. Upon receiving a query, KGARevion generates relevant triplets by leveraging the latent knowledge embedded in a large language model. It then verifies these triplets against a grounded knowledge graph, filtering out errors and retaining only accurate, contextually relevant information for the final answer. This multi-step process strengthens reasoning, adapts to different models of medical inference, and outperforms retrieval-augmented generation-based approaches that lack effective verification mechanisms. Evaluations on medical QA benchmarks show that KGARevion improves accuracy by over 5.2% over 15 models in handling complex medical queries. To further assess its effectiveness, we curated three new medical QA datasets with varying levels of semantic complexity, where KGARevion improved accuracy by 10.4%. The agent integrates with different LLMs and biomedical knowledge graphs for broad applicability across knowledge-intensive tasks. We evaluated KGARevion on AfriMed-QA, a newly introduced dataset focused on African healthcare, demonstrating its strong zero-shot generalization to underrepresented medical contexts.

1 INTRODUCTION

Medical QA requires combining specialized biomedical knowledge with context, but LLMs and retrieval-based approaches can miss, misinterpret, or fail to verify relevant evidence. KGAREVION generates and checks knowledge-graph triplets, and the paper reports improved performance across its evaluated benchmarks.

1 INTRODUCTION

  • LLMs can struggle with specialized medical concepts, interrelated dependencies, and semantically similar but distinct biomedical entities.Biomedical reasoning also requires integrating codified scientific knowledge with tacit clinical expertise and local context.Source: §1 · pp. 1–2
  • RAG answers depend on retrieved-content quality, and many approaches lack post-retrieval checks of factual accuracy, relevance, or omissions.Biomedical knowledge sources can be incomplete, outdated, or incorrect.Source: §1 · p. 2
  • Knowledge graphs provide specialized grounded knowledge but can have gaps, while direct-edge retrieval can miss implicit or complex biomedical relationships.The introduction identifies a need to combine multi-concept associations with systematic knowledge integration and verification.Source: §1 · p. 2
  • KGAREVION generates question-relevant triplets, uses a fine-tuned LLM to evaluate them, revises errors, and adapts its reasoning approach to the question.The approach supports both multiple-choice and open-ended QA.Source: §1 · p. 2
  • 6.75% improvement over 15 baseline models across seven datasets is reported for KGAREVION, including three newly curated challenging benchmarks.The paper also reports zero-shot generalization on AfriMed-QA and robustness to answer-choice reordering.Source: §1 · pp. 2–3

2 RELATED WORK

Prior medical QA approaches use LLM reasoning, retrieval-augmented generation, or knowledge graphs, but each has limitations in knowledge coverage or quality. The paper also reviews graph-based retrieval and grounding methods.

LLM-based reasoning

  • LLM reasoning methods, including Chain-of-Thought and CODEX CoT, can struggle with knowledge-intensive queries requiring multi-source, domain-specific knowledge.The passage describes general-purpose and biomedical-fine-tuned LLMs as leveraging embedded knowledge, with some methods decomposing queries into subtasks.Source: §2 · p. 3

RAG-based models

  • RAG-based approaches add retrieval or self-refinement, but performance can remain restricted by the quality of retrieved knowledge.The examples include Self-RAG, LLM-AMT, and Adaptive-RAG.Source: §2 · p. 3

KG-based models

  • KG-only QA models can struggle with unseen nodes or incomplete graphs, while graph-RAG methods may overlook rich KG structure when relying heavily on semantic dependencies.The passage names QAGNN, JointLK, Dragon, GraphRAG, KG-RAG, MedGraphRAG, KG-Rank, and GenGround.Source: §2 · p. 3

3 KGAREVION AGENT

KGAREVION combines LLM-generated biomedical knowledge with knowledge-graph validation through Generate, Review, Revise, and Answer actions. Its procedures vary by question type and use graph-informed checking before producing an answer.

3 KGAREVION AGENT

  • KGAREVION uses four actions: Generate relevant triplets, Review their correctness, Revise incorrect triplets, and Answer from triplets verified by Review.The agent is framed as combining LLM-generated knowledge with structured KG-based validation.Source: §3 · p. 3

3.1 GENERATE ACTION

  • For choice-aware questions, Generate produces triplets using each answer candidate and stem concepts; for yes/no/maybe questions, it uses stem concepts directly.The design is motivated by LLM familiarity biases and reducing sensitivity to answer-candidate order.Source: §3.1 · p. 4

3.2 REVIEW ACTION

  • Review uses a fine-tuned LLM that incorporates KG entity embeddings to assess generated triplets beyond semantic dependencies inferred by the LLM.The model is fine-tuned on a KG completion task.Source: §3.2 · p. 4

3.2.1 FINE-TUNING STAGE

  • The fine-tuning stage aligns fixed TransE entity and relation embeddings with relation descriptions, then uses the aligned triplet embedding as an instruction prefix during LLM fine-tuning.The alignment combines mapped head, relation, and tail embeddings with the LLM embedding of the relation description; the LLM is fine-tuned with LoRA and next-token prediction loss to judge triples True or False.Source: §3.2.1 · p. 5

3.2.2 INFERENCE STAGE

  • At inference, mapped entities are checked by the fine-tuned LLM; a mapped triplet judged False is removed, while a triplet with an unmapped head or tail is retained as incomplete knowledge.The procedure groups reviewed triplets into True and False sets.Source: §3.2.2 · p. 5

3.3 REVISE AND ANSWER ACTIONS

  • Revise adjusts false triplets to cover more medical concepts, and Review checks the replacements until accepted or the maximum revision round is reached; Answer uses verified triplets.The process calls Answer after obtaining the True triplet set.Source: §3.3 · pp. 5–6

4 RESULTS

The results evaluate medical QA across four gold-standard benchmarks and three MedDDx datasets, using multi-choice and open-ended settings and tests of concept-count and semantic complexity.

4.1 MULTI-CHOICE QUESTION-ANSWERING

  • At n = 6, KGAREVION can improve over n = 5 while baselines struggle at 5–6 concepts; MedDDx-Expert accuracy improves 3.2% with LLaMA3-8B and 5.6% with LLaMA3.1-8B.The concept-count comparison is shown in Figure 3a.Source: §4.1 · pp. 6–7, Figure 3 · p. 8
  • Over 4.8% average-accuracy improvement across four gold-standard medical QA datasets, with KGAREVION outperforming all baselines.Table 2 covers accuracy and variance under multi-choice reasoning.Source: §4.1 · p. 6, Table 2 · p. 7

4.2 OPEN-ENDED REASONING

  • Removing answer choices tests generated responses; KGAREVION remains best in open-ended CSS and outperforms pure LLMs above four concepts, with comparable performance below three.The open-ended setting masks candidate answers so the model must generate a response.Source: §4.2 · pp. 7–8, Table 3 · p. 8

4.3 ABLATION ANALYSES

  • Review improves average accuracy by 9% in multi-choice and 4% in open-ended reasoning, with a larger effect on MedDDx than on the other four datasets.The authors note greater Review gains on gold-standard datasets under open-ended reasoning.Source: §4.3 · p. 8, Figure 4 · p. 9
  • Revise improves MedDDx accuracy by 3.3% in multi-choice and 3.0% in open-ended reasoning; k = 1 is optimal for multi-choice, while MedDDx-Expert benefits from more rounds.Open-ended reasoning typically requires more iterations to refine and verify the correct answer.Source: §4.3 · p. 9, Figure 4 · p. 9

4.4 VERSATILITY OF KGAREVION

  • Across backbones, KGAREVION raises accuracy by 6% with LLaMA3-8B, 7% with LLaMA3.1-8B, and 2% with GPT-4-Turbo; results remain high with either tested KG.The authors attribute KG stability to using graphs for triplet verification rather than primary retrieval.Source: §4.4 · p. 9, Figure 5 · p. 9

4.5 EVALUATION OF KGAREVION ON AFRICAN HEALTHCARE

  • On AfriMed-QA, KGAREVION improves accuracy by 5.2% with LLaMA3.1-8B and 4.6% with GPT-4-Turbo, showing zero-shot generalization to an unseen African-healthcare dataset.The evaluation uses expert-level multiple-choice questions.Source: §4.5 · pp. 9–10, Figure 6 · p. 10

4.6 SENSITIVITY ANALYSES

  • KGAREVION reduces answer-order sensitivity and cuts relabeling shifts to 2.59% for LLaMA3-8B and 3.86% for LLaMA3.1-8B, versus LLM-only shifts of 8.1% and 12.9%.Figure 7 reports accuracy changes when answer order or indexing is changed.Source: §4.6 · p. 10, Figure 7 · p. 10

5 CONCLUSION

KGAREVION combines LLM knowledge with medical knowledge graphs to address biomedical QA requiring specialized, grounded reasoning. The paper reports improved accuracy across multiple-choice and open-ended tasks, including newly curated benchmarks.

5 CONCLUSION

  • KGAREVION combines LLM knowledge with structured medical knowledge graphs to generate, verify, and refine knowledge for biomedical QA.The conclusion describes this combination as a way to adapt reasoning to biomedical QA complexity.Source: §5 · p. 10
  • Evaluations across multiple-choice and open-ended tasks, including newly curated challenging benchmarks, show consistent accuracy improvements over existing methods.Source: §5 · p. 10
  • The authors identify potential applications in clinical decision support, biomedical research, and complex medical inference.Source: §5 · p. 10

ETHICS STATEMENT

The ethics statement urges caution about real-world medical use of LLMs and calls for human oversight. The extended related work reviews rule-based, graph-based, and retrieval-based approaches to fact-checking generated text.

ETHICS STATEMENT

  • LLM hallucinations and dataset biases may contribute to inaccurate advice and disparities in healthcare recommendations.The statement also identifies transparency, interpretability, and fairness as ongoing challenges.Source: ETHICS STATEMENT · p. 11

A EXTENDED RELATED WORK

  • Fact-checking approaches reviewed include manually defined rules, KG connectivity or completion, and RAG-based external evidence retrieval.Source: A EXTENDED RELATED WORK · p. 18

B.1 GOLD STANDARD MULTI-CHOICE MEDICAL QA DATASET

The appendix describes four established medical QA datasets and AfriMed-QA, alongside MedDDx, a new benchmark designed around semantic complexity. It also outlines the conversion of multiple-choice questions into descriptive, open-ended questions.

B.1 GOLD STANDARD MULTI-CHOICE MEDICAL QA DATASET

  • Four established multi-choice benchmarks cover medical examination QA and biomedical research QA.The examination datasets are MMLU-Med and MedQA-US; the research datasets are PubMedQA* and BioASQ-Y/N.Source: §B.1 · p. 18

B.2 MEDDDX

  • MedDDx tests semantically complex answer choices, using similar alternatives and standard-deviation quantiles to group questions into three complexity levels.The groups are MedDDx-Expert (0-0.02), MedDDx-Intermediate (0.02-0.04), and MedDDx-Basic (>0.04).Source: §B.2 · pp. 18–19

B.3 AFRIMED-QA

  • AfriMed-QA contains 15,275 pan-African clinical QA from over 500 contributors across 16 countries and 32 specialties.Its questions and answers are described as absent from KGARevion’s LLM and KG.Source: §B.3 · p. 19

B.4 CONVERSION OF MULTI-CHOICE TYPE QUESTIONS TO DESCRIPTIVE TYPE

  • Multiple-choice questions are converted into more descriptive questions to evaluate open-ended reasoning in scenarios where such inquiries are prevalent.Source: §B.4 · p. 20

C.1 EXPERIMENT ENVIRONMENTS

The experiments used specified hardware and software environments, fine-tuned the review module for KG completion, and averaged reported results over three random seeds.

C.1 EXPERIMENT ENVIRONMENTS

  • Experiments ran on a machine with four NVIDIA H100 GPUs, with one GPU used for small-LLM baselines and four for fine-tuning the review module.Source: §C.1 · p. 20

C.2 FINE-TUNING DETAILS

  • The review module was LoRA-fine-tuned on PrimeKG KG-completion data split 8:2, with r = 32 and batch size 256 selected by grid search.Source: §C.2 · p. 21

C.3 IMPLEMENTATION DETAILS OF BASELINE MODELS

  • Reported results are averages over random seeds 42, 777, and 1234.Source: §C.3 · p. 21

C.3.1 IMPLEMENTATION DETAILS OF BASELINE MODELS UNDER MULTI-CHOICE SETTING

The appendix specifies baseline implementations for multi-choice and open-ended evaluation, then describes tests of sensitivity to answer order and indices.

C.3.1 IMPLEMENTATION DETAILS OF BASELINE MODELS UNDER MULTI-CHOICE SETTING

  • Baseline LLMs were evaluated with traditional and chain-of-thought inference using Transformers 4.31.1 and default parameters.KG baselines were trained on 4,182 MedMCQA samples and evaluated using PrimeKG subgraphs; RAG baselines used the specified backbones, corpus, and retriever configurations.Source: §C.3.1 · p. 21

C.3.2 IMPLEMENTATION DETAILS OF BASELINE MODELS UNDER OPEN-ENDED REASONING SETTING

  • Open-ended evaluation added a response-generation module for LLM- and RAG-based models, while KG-based models could not be deployed in this setting.For the other two model groups, responses were matched to correct-answer content without considering the input question.Source: §C.3.2 · p. 21

C.4.1 EFFECT OF ANSWER ORDER/INDEX

  • Answer-order sensitivity was tested by swapping options, and index sensitivity by relabeling answer choices.The altered orders were ABCD→BCAD, ABC→CAB, or AB→BA; four-option indices were relabeled ABCD→EFGH. Table 7 presents example questions.Source: §C.4.1 · p. 22, Table 7 · p. 22

C.5 IMPLEMENTATION DETAILS OF ADAPTABILITY ANALYSIS

The adaptability analysis tests KGARevion with two knowledge graphs, PrimeKG and OGB-biokg, whose biomedical coverage is described here.

C.5 IMPLEMENTATION DETAILS OF ADAPTABILITY ANALYSIS

  • Adaptability analysis tested KGARevion with PrimeKG and OGB-biokg.PrimeKG is described as precision-medicine-oriented, while OGB-biokg contains five entity types and 51 directed relation types.Source: §C.5 · p. 22
  • PrimeKG integrates 20 resources and represents 17,080 diseases with 4,050,249 relationships across ten biological scales.Its coverage includes disease-associated protein perturbations, pathways, anatomy, phenotypes, and approved and experimental drugs.Source: §C.5 · p. 22
  • OGB-biokg includes diseases, proteins, drugs, side effects, and protein functions connected by 51 directed relation types.The listed relations include 38 drug-drug interaction types and eight protein-protein interaction types.Source: §C.5 · p. 22

D.1 IMPLEMENTATION DETAILS IN REVISE ACTION

The appendix details how KGARevion revises false triplets and uses verified triplets to produce answers, alongside prompt templates and supplementary reference material.

D.1 IMPLEMENTATION DETAILS IN REVISE ACTION

  • False triplets are revised by an instructed, unfine-tuned LLM, which changes a head or tail entity to produce a query-relevant correction.A revised triplet is checked again; true triplets proceed to answering, while false ones return for revision until accepted or the round limit is reached.Source: §D.1 · pp. 22–23

D.2 IMPLEMENTATION DETAILS IN ANSWER ACTION

  • The Answer action prompts an unfine-tuned LLM to generate the final answer from triplets verified as true.The appendix says this uses pre-trained models without additional training.Source: §D.2 · p. 23

E.1 PROMPT TEMPLATE FOR EVALUATING BASELINE MODELS UNDER MULTI-CHOICE REASONING SETTING

  • The multi-choice baseline prompt asks models to select the correct answer from options A–D, with a separate version prompting step-by-step reasoning.The open-ended template asks for a response to the medical question, while its CoT version adds “Let’s think step by step.”Source: §E.1 · p. 23, §E.2 · p. 24

E.3 PROMPT TEMPLATE IN KGAREVION

  • KGARevion’s Generate prompts first extract medical entities, then generate relevant undirected triplets for each option.The Review prompt classifies a triplet as true or false; the Revise prompt requests a factually accurate, question-relevant triplet.Source: §E.3 · pp. 24–26

F.1 DESCRIPTION TEMPLATE; F.2 NOTATIONS

  • The appendix includes a description-template table and a table of additional notation.The supplied passage titles identify these references but do not specify their contents.Source: Table 8 · p. 27, Table 9 · p. 28

Related papers

Loading 2410.04660v2…