HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses

1 INTRODUCTION

• Medical LLMs remain vulnerable to factual inaccuracies and limited explainability, alongside data constraints, catastrophic forgetting, outdated knowledge, and limited expertise on specialized queries.These issues undermine reliability where accountability and trustworthiness matter.Source: §1 · p. 1
• KG-based RAG offers structured external knowledge, but inconsistent user queries make intent parsing and management of abundant retrieved knowledge difficult.KGs provide succinct structured content and support analysis of relationships among entities.Source: §1 · p. 1
• Prior approaches face three related challenges: noisy or redundant retrieval from unclear queries, costly multi-round LLM interactions, and repetitive results when reranking uses sparse queries alone.The reported costs include token waste, slower responses, time overhead, and cumulative reasoning errors.Source: §1 · pp. 1–2
• HyKGE uses LLM hypothesis outputs to expand retrieval directions, prompts to elicit comprehensive hypotheses with fewer interactions, and HO fragments to rerank chains while retaining relevant, diverse knowledge.The authors describe experiments on three medical Q&A tasks and potential applications in consultation quality, diagnosis accuracy, and medical research.Source: §1 · p. 2
• HyKGE is reported to outperform baselines on two Chinese medical multiple-choice datasets and one open-domain medical Q&A dataset, with accuracy and explainability as target benefits.The experiments use two LLM turbos.Source: §1 · p. 2

2 RELATED WORK

• Classic RAG retrieves documents and reranks them, while training query-document retrievers can be limited by scarce high-quality training data or sparse user queries.Fine-tuning reader LLMs can require manual label correction, which may erode generalization and cause catastrophic forgetting in routine Q&A.Source: §2 · p. 2
• HyDE generates hypothesis documents to improve retrieval, whereas CoN and CoK use customized prompts for note-making and stepwise verification but rely on frequent LLM interactions.The passage characterizes that interaction burden as inefficient for real-world Q&A deployment.Source: §2 · p. 2
• HyKGE uses LLM hypothesis outputs to explore KG directions and graph reasoning chains to correct errors, then aligns retrieved chains at finer granularity in post-retrieval.The approach is described as maintaining effective, diverse knowledge without fine-tuning or excessive interactions.Source: §2 · pp. 2–3
• KGs offer structured, inferable knowledge, but KGRAG can misalign with variable-quality queries and semantic parsing can falter on intricate queries.Semantic parsing methods generate structural queries such as SPARQL for KG execution.Source: §2 · p. 3

3 PRELIMINARIES

• A medical KG is represented by entities, relations, and head-relation-tail triplets, with descriptions attached to entities.The definition also identifies neighboring relations and entities for each entity.Source: §3 · p. 3
• KG retrieval seeks relevant information from a KG based on a user query, finding entities or relationships that can supply knowledge to LLMs.The described retrieved material may include entity names, types, descriptions, triplets, or subgraphs.Source: §3 · p. 3
• The retrieved subgraph for a matched entity can be represented using that entity, its triplets, and its description.The definition denotes this structure as G_ej = (e_j, T_j, d_j).Source: §3 · p. 3

4 METHOD

• The pre-retrieval phase uses an LLM to generate hypothesis output, then extracts medical entities from both that output and the user query.The NER Module recognizes entities and isolates relations.Source: §4 · p. 3, Figure 2 · p. 4
• Recognized entities are linked to KG entities, which serve as anchors for retrieving three types of reasoning chains between anchors.The chains provide relevant and logical knowledge.Source: §4 · p. 3, Figure 2 · p. 4
• The method segments the query and hypothesis output into fragments, reranks retrieved chains against those fragments, and sends the pruned chains with the query to the LLM Reader.The reranking step is described as filtering irrelevant knowledge before answer generation.Source: Figure 2 · p. 4, §4 · p. 4

4.1 Pre-Retrieval Phase

• The NER module extracts entities from both the query and hypothesis output while ignoring their relations, letting graph retrieval recover relationships and avoid relying on potentially hallucinated links.The NER model was trained using the CMEEE dataset.Source: §4.1 · p. 4, §4.1.2 · p. 4

4.1.1 Hypothesis Output Module.

• The Hypothesis Output Module prompts LLMs to explore queries step by step, producing hypothesis outputs with relevant medical knowledge for retrieval.The prompt is denoted PHO, and the output is HO.Source: §4.1.1 · p. 4

4.1.2 NER Module.

• The prompt-format figure presents the Hypothesis Output Module format above and the LLM Reader format below.The NER model is built on W2NER, which addresses flat, overlapped, and discontinuous NER cases.Source: Figure 3 · p. 5, §4.1.2 · p. 5
• The NER input concatenates the user query and hypothesis output, and its outputs are extracted medical entities.The paper denotes concatenation with ⊕ and each extracted entity with u_i.Source: §4.1.2 · p. 5

4.2.1 Embedding Alignment.

• KG entity linking encodes each extracted entity and KG entities, then matches the highest-similarity entity when its score exceeds threshold δ.The paper uses GTE embeddings and defines δ within [0, 1].Source: §4.2.1 · p. 5

4.2.2 Search Reasoning Chains in Knowledge Graph.

• Reasoning chains between matched entities provide richer logical context than entities or descriptions alone and are described as helping the reader understand relationships, reduce hallucinations, and prune noise.The paper identifies path, co-ancestor, and co-occurrence chain types for analyzing disease-symptom relations, analogous factors, and disease characteristics, respectively.Source: §4.2.2 · p. 5
• For each matched entity pair, HyKGE collects reasoning chains within k hops, where k is at least 2, and the head and tail entity descriptions.Source: §4.2.2 · p. 6

• HyKGE uses the bge_reranker_large model to prune noisy reasoning chains, improving token-resource use after retrieval.The paper motivates reranking by the volume of retrieved chains, noise, and limited token resources.Source: §4.3 · p. 6
• The reranker uses fragments from both HO and Q rather than Q alone, because query-only reranking may discard HO-derived knowledge and yield repetitive results.The method removes stop words and chunks HO and Q before reranking.Source: §4.3 · p. 6
• The reranker evaluates retrieved chains against the fragment set for relevance, coherence, and informativeness, then retains the top K chains.The number retained is |RCprune| = topK.Source: §4.3 · p. 6

• HyKGE links the pruned reasoning chains with directed arrows and head- and tail-entity descriptions, then prompts the LLM Reader with that knowledge and the user query.Source: §4.4 · p. 6

4.5 HyKGE Process

• HyKGE embeds KG entity names locally, obtains a hypothesis output from the user query, and extracts entities from both the hypothesis and query.It matches those entities to the KG, retrieves reasoning chains between anchor entities, prunes noise, then prompts an LLM reader with the query and remaining knowledge.Source: §4.5 · p. 6
• The pipeline uses the HO Fragment Granularity-aware Rerank Module to filter noisy reasoning chains before organizing retrieved knowledge with the query for the LLM reader.The reader then produces the optimized answer.Source: §4.5 · p. 6

5.2 Performance Comparison (RQ 1)

• HyKGE outperforms all baselines across metrics; on MMCU-Medical with GPT 3.5 turbo, EM and PCR improve approximately 8.55%-28.15% and 10.45%-33.29%, respectively.The reported comparison also finds superior performance over baselines with fewer LLM interactions.Source: Table 2 · p. 7, §5.2 · pp. 8–9

5.1 Experimental Setup

• The evaluation covers MMCU-Medical and CMB-Exam medical question sets and CMB-Clin open-domain Q&A, including 4,000 sampled CMB-Exam questions and 208 questions across 74 CMB-Clin cases.MMCU-Medical contains 2,819 questions.Source: §5.1.1 · pp. 6–7

5.1.2 Knowledge Graph.

• The fused medical KG combines CMeKG, CPubMed-KG, and Disease-KG, totaling 1,288,721 entities and 3,569,427 relations, with entity descriptions collected from online encyclopedias.The descriptions supplement the KG's entities, which lacked medical entity descriptions.Source: §5.1.2 · p. 7

5.1.3 LLM Turbo.

• HyKGE is tested with GPT 3.5 and Baichuan13B-chat against eight alternatives, including KGRAG, query expansion, and several LLM-based retrieval or reasoning methods.CoN requires retriever fine-tuning; several compared methods, including CoN, CoK, KALMV, KG-GPT, and SuRe, require more than two LLM interactions.Source: §5.1.3 · p. 7, §5.1.4 · pp. 7–8

5.1.5 Evaluation Metrics.

• Choice-task evaluation uses EM and PCR, while retrieved-knowledge relevance and answer quality are assessed with ACJ, PPL, ROUGE-R, and open-domain BLEU-1/BLEU-4.ACJ uses ratings from 20 medical experts on sampled questions; experiments are repeated with five random seeds and report averages and standard deviations.Source: §5.1.5 · p. 8, §5.1.6 · p. 8

5.3 Ablation Study (RQ 2)

• Removing the Hypothesis Output Module makes results worse than the base model, while reranking reasoning chains still leaves results better than KGRAG.The authors attribute this to query-only retrieval being insufficient or directionless, with reranking filtering noise.Source: §5.3 · p. 9
• Removing the reranker reduces performance, and reranking with entire HO and Q instead of fragments also declines performance because dense knowledge misaligns with sparse query keywords.The mismatch favors more general or lengthy knowledge and diminishes the HOM's ability to supplement diverse knowledge.Source: §5.3 · p. 9
• Ablations indicate that descriptions or relevant knowledge chains can still enhance LLM answers when the KG lacks some knowledge.The authors associate this with the LLMs' inherent implicit knowledge.Source: §5.3 · p. 9

5.4 Interpretability Analysis (RQ 3)

• LLM-interacting methods with noise filtering often score higher on ACJ; KG-GPT and QE are second-best because they preserve the user query’s semantics.The authors contrast these methods with HyKGE’s gains in solving misaligned knowledge density.Source: §5.4 · p. 9
• 84.19%-378.71% on MMCU-Medical and 133.33%-345.22% on CMB-Exam: HyKGE surpasses baselines on ACJ; BLEU and ROUGE-R on CMB-Clin also demonstrate HyKGE’s superiority.The authors attribute the gains to better alignment between user queries and retrieved knowledge.Source: §5.4 · p. 9
• 57.77%-95.36% on MMCU for PPL and 61.03%-185.42% on MMCU for ROUGE-R: HyKGE improves LLM use of retrieved knowledge and reduces hallucinations.The authors associate the result with relevant, diverse knowledge from HOM and the HO Fragment Granularity-aware Reranker.Source: §5.4 · p. 9

5.5 Hyper-parameter Study (RQ4)

• As k rises, retrieved knowledge exceeds 10^3 at k=5 while EM first rises then falls; higher topK also hampers comprehension, and HyKGE w/o Fragment underperforms on EM.The authors report that removing HO substantially reduces retrieved knowledge because Q alone offers limited diversity.Source: §5.5 · pp. 9–10, Figure 5 · p. 10

5.6 Case Study (RQ2 and RQ3)

• HO yielded 23 entities versus 1 from Q, and reranking selected 10 of 125 reasoning chains after removing irrelevant or repetitive knowledge.Figure 4 marks Q-derived evidence red, HO-derived evidence blue, and KG-corrected answers green.Source: §5.6 · pp. 10–11, Figure 4 · p. 10
• In the case study, retrieved knowledge about calcium carbonate helped the LLM correct its initially false HO claim that it could not treat GERD.The authors present this as an example of using hypotheses to explore KG reasoning chains and correct errors.Source: §5.6 · p. 11

5.7 Efficiency Analysis (RQ2)

• HyKGE is reported as the most efficient and high-performing framework; filtering adds slightly more time than QE, while CoN and CoK require restrictive multiple LLM interactions.The authors say HyKGE’s performance enhancement outweighs its marginal time increase over QE.Source: §5.7 · p. 11
• Using LLMs for NER, reranking, or summarization incurred substantial time or token costs with marginal benefits, and LLM reranking could remove valuable knowledge.For the reflux query, reranking 125 reasoning chains with an LLM decreased effectiveness and increased computational overhead.Source: §5.7 · p. 11

6 CONCLUSION

• HyKGE uses LLM zero-shot capability to compensate for incomplete queries with hypothesis-guided KG search, then applies fragment reranking to preserve relevant and diverse knowledge chains.The reranker is intended to improve knowledge-density alignment between user queries and retrieved knowledge.Source: §6 · p. 11
• Experiments on three medical Q&A tasks with two LLM turbos demonstrate HyKGE’s effectiveness.Source: §6 · p. 11
• Dynamic fragment-granularity optimization remains future work; the authors also cite data-source limitations and high LLM computational cost while proposing broader KG evaluation to enhance scalability and generalization.Source: §6 · p. 11