Source-linked AI summary
MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, Yihan Cao, Hui Ren, Xiang Li, Xiaoxiao Li, Yuyin Zhou
TL;DR
Medical reasoning needs transparent, verifiable step-by-step data, but high-quality medical CoT is scarce, difficult to scale, and vulnerable to factual errors from general-purpose models. MedReason uses a medical knowledge graph to guide and filter CoT generation, producing 32,682 samples and improving performance across medical benchmarks, including a 4.2% advantage over Huatuo-o1-RL-8B on MedBullets.
Problem
High-quality, scalable medical Chain-of-Thought data is scarce, while general-purpose LLMs can introduce factual errors into medical reasoning.
Method
MedReason maps Q&A entities to a medical knowledge graph, uses KG reasoning paths as factual guidance, and filters generated CoT by answer correctness.
Results
MedReason fine-tuning consistently improves performance across seven medical QA benchmarks, with MedReason-8B outperforming Huatuo-o1-RL-8B by 4.2% on MedBullets.
Takeaways & Limitations
Knowledge-guided CoT data produces medically grounded and interpretable explanations that improve reasoning, particularly on complex clinical scenarios.
Takeaways & Limitations
General-purpose LLMs used to generate medical CoT can produce factual errors, as illustrated by an incorrect treatment conclusion for ARDS.
Abstract
from arXiv · showhide
Medical tasks such as diagnosis and treatment planning require precise and complex reasoning, particularly in life-critical domains. Unlike mathematical reasoning, medical reasoning demands meticulous, verifiable thought processes to ensure reliability and accuracy. However, there is a notable lack of datasets that provide transparent, step-by-step reasoning to validate and enhance the medical reasoning ability of AI models. To bridge this gap, we introduce MedReason, a large-scale high-quality medical reasoning dataset designed to enable faithful and explainable medical problem-solving in large language models (LLMs). We utilize a structured medical knowledge graph (KG) to convert clinical QA pairs into logical chains of reasoning, or ``thinking paths'', which trace connections from question elements to answers via relevant KG entities. Each path is validated for consistency with clinical logic and evidence-based medicine. Our pipeline generates detailed reasoning for various medical questions from 7 medical datasets, resulting in a dataset of 32,682 question-answer pairs, each with detailed, step-by-step explanations. Experiments demonstrate that fine-tuning with our dataset consistently boosts medical problem-solving capabilities, achieving significant gains of up to 7.7% for DeepSeek-Ditill-8B. Our top-performing model, MedReason-8B, outperforms the Huatuo-o1-8B, a state-of-the-art medical reasoning model, by up to 4.2% on the clinical benchmark MedBullets. We also engage medical professionals from diverse specialties to assess our dataset's quality, ensuring MedReason offers accurate and coherent medical reasoning. Our data, models, and code is available at https://github.com/UCSC-VLAA/MedReason.
1 Introduction
MedReason addresses the scarcity and limited scalability of high-quality medical Chain-of-Thought data by using knowledge graphs to guide and verify reasoning. Its resulting dataset improves medical problem-solving across instruction-tuned and reasoning-specialized models.
- High-quality medical CoT data remains scarce and difficult to scale, limiting the development of stronger medical reasoning models.
- Knowledge graphs provide factual guidance intended to preserve logical coherence and clinical validity throughout medical reasoning steps.
- The pipeline verifies generated reasoning by retaining only CoT samples whose reasoning produces the correct answer.
- 32,682 high-quality CoT samples were generated from medical question-answer pairs and showed consistently improved reasoning quality across evaluation metrics.
- Evaluation covered seven QA benchmarks, including common medical and challenging clinical benchmarks, across instruction-tuned and medical reasoning models.
2 Related Works
Prior medical reasoning datasets often distill intermediate reasoning from general-purpose language models, but maintaining medical factual integrity at every step remains challenging. MedReason instead uses a medical knowledge graph to provide dependable factual guidance for generating medical CoT data.
- Reasoning models require large quantities of high-quality data containing intermediate reasoning steps, while manual annotation is not scalable.
- Its pipeline extracts and maps Q&A entities, searches and prunes KG reasoning paths, uses them to construct CoT data, and filters incorrect generated reasoning.
- Directly using general-domain LLMs to generate medical CoT can challenge the integrity of medical knowledge at every reasoning step.
- MedReason employs a medical knowledge graph to provide factual guidance during medical CoT generation.
3 Method
MedReason retrieves medically relevant reasoning paths by mapping question and answer entities to PrimeKG, selecting concise question-related paths, and using them to guide grounded CoT generation. Generated explanations are then answer-checked, retaining only instances that reproduce the ground-truth answer.
- Medical Entity Extraction and Mapping: GPT-4o identifies medical entities in each question and answer, which are mapped to nodes in the PrimeKG knowledge base.The mapping uses text embeddings to rank candidate graph entities, followed by exact, similarity-based, or LLM-based selection.
- Medical Entity Extraction and Mapping: Entity mapping first accepts exact matches, then uses a similarity threshold of 0.85, and finally invokes LLM-based selection when needed.The final stage evaluates the question-answer context and entity name against candidate nodes.
- Paths Searching and Pruning: Shortest paths connect mapped question entities to answer entities, while LLM pruning retains paths most correlated with the question.The procedure uses shortest paths to maintain concise reasoning and filters irrelevant alternatives before CoT generation.
- CoT Generation: The pipeline aggregates the selected paths and prompts an LLM to elaborate relevant ones into medically grounded, clinically aligned CoT explanations.The resulting chains can proceed from symptom analysis through pathological deduction to a final diagnosis, with each step aligned to KG evidence.
- Quality Filtering: 32,000 of 45,000 generated samples were retained after an LLM used each CoT to produce an answer that matched the ground truth.This filtering step is intended to preserve logical validity and factual accuracy in the final dataset.
4 Experiment
Experiments evaluate MedReason across instruction-tuned and reasoning models, medical benchmarks, ablations, and expert verification. Fine-tuning generally improves performance, while quality filtering and expert judgments support the dataset’s reasoning quality.
- Data Collection: 55K training QA pairs were curated from seven medical datasets, with training-only generation used to prevent data leakage.HLE and MedXpert used designated training samples with the remainder reserved for testing.
- Instruction-Tuned Models: Llama3.1-Instruct-8B accuracy increased from 45.8% to 51.2% (+5.4%) with MedReason, exceeding Huatuo CoT’s +4.1% gain.
- Instruction-Tuned Models: Mistral-Instruct-7B accuracy increased from 36.1% to 44.7% (+8.6%), exceeding Huatuo CoT’s +6.0% gain.
- Reasoning Models: DeepSeek-Distill-8B gained 7.7% on challenging clinical datasets, while Medical-CoT-8B gained 5.3%.MedReason improved reasoning models across clinical and general medical question-answering tasks.
- Benchmark Comparison: MedReason-8B outperformed Huatuo-o1-RL-8B by 4.2% on MedBullets (op5) and by 2.3% on MedXpert.It also showed approximately 12% gains over OpenBioLLM-8B and DeepSeek-Distill-8B.
- Ablation Study: Quality filtering improved the average score by 1.1% across the majority of medical datasets.The filtering retained reasoning samples that enabled correct answers.
- Expert Verification: Experts unanimously preferred MedReason in Gastroenterology (100%), with Dermatology and Oncology preferences exceeding 80%.The verification covered seven medical specialties and compared MedReason with Huatuo-Complex CoT.
5 Conclusion
The paper presents a scalable knowledge-guided pipeline for generating medically grounded, interpretable CoT data. Across benchmarks and model types, it reports consistent improvements, especially in complex clinical scenarios, alongside stronger expert-rated reasoning quality.
- MedReason uses structured knowledge graphs to anchor automatically generated medical CoT explanations in clinical facts.The paper frames this as supporting clinical validity and interpretability.
- Experiments report consistent benchmark improvements across instruction-tuned and reasoning-specialized LLMs, particularly for complex clinical scenarios.
- Expert evaluations favored MedReason’s reasoning quality over prior methods across diverse medical specialties.
A.1 OpenAI API Usage
The appendix reports the OpenAI API configuration and total usage cost for the data-generation pipeline.
- The pipeline used Azure GPT-4o version gpt-4o-0806-nofilter-global with API version 2024-12-01-preview.
- Total GPT-4o API usage cost approximately $3,600.
A.2 Data Statistics
MedReason’s data statistics describe dataset-specific training and testing allocations and the use of training data for generation.
- HLE allocated 57 samples and MedXpert allocated 666 samples for training, reserving the remaining data for testing.
- Other source datasets used only their training data for CoT generation to avoid potential data leakage.
- Table 6 reports statistics for the raw, generated, and final quality-filtered QA datasets.
A.3 Prompts
The data-generation pipeline uses dedicated prompts to identify entities, select relevant knowledge-graph nodes, and prune irrelevant reasoning paths.
- The pipeline identifies entities in each question and answer before constructing knowledge-grounded reasoning data.
- It selects the most relevant knowledge-graph nodes for the extracted entities.
- It prunes irrelevant paths to retain reasoning paths relevant to the question-answer pair.
A.4 Detailed Algorithm
The algorithm maps extracted entities to knowledge-graph nodes through exact matching, similarity matching, or LLM-based selection, then supports downstream reasoning-path generation and answer filtering.
- Entity mapping: The algorithm encodes extracted entities and graph nodes, ranks candidate matches, and forms a Top-K candidate set.A text embedding model computes similarities between each entity and node embeddings in the knowledge graph.
- Entity mapping: Exact matching selects a graph node when an entity matches a candidate after lowercasing.
- Entity mapping: If exact and similarity matching fail, an LLM analyzes the question-answer context and candidate names to select a node.
- Entity mapping: The mapping can connect a clinical phrase such as bilateral optic disc swelling to the related graph concept Abnormality of the optic disc.
- Entity mapping: Similarity matching selects the most similar candidate when no exact match exists and the top score exceeds τ = 0.85.
A.5 Detailed comparison between MedReason-8B and Huatuo-o1-RL-8B.
The comparison examines MedReason-8B and Huatuo-o1-RL-8B on complex clinical issues, highlighting a case where MedReason-8B reaches the correct answer through factual reasoning.
- The comparison evaluates MedReason-8B and Huatuo-o1-RL-8B on complex clinical issues.
- Huatuo-o1-RL-8B gives an incorrect answer after overlooking the link between vomiting and hyperglycemia.
- MedReason-8B identifies the correct answer using factual reasoning.