Source-linked AI summary
Medical Causal Hypothesis Verification with Large Language Models
Safiyyah Ahmed, Abrar Ansari, Md Aminul Islam, Elena Zheleva
TL;DR
This preliminary study evaluates whether LLMs can verify medical causal hypotheses and support their conclusions with scientific evidence. Across the evaluated models, recall was generally high, but rejecting false claims and providing valid, traceable sources remained difficult.
Problem
Existing research had not evaluated whether LLMs could retrieve scientific evidence to substantiate causal medical claims, despite the importance of caution in high-stakes settings.
Method
The study evaluates eight LLMs on 17 clinician-validated medical causal hypotheses, asking them to classify each hypothesis and provide up to three supporting scientific articles when appropriate.
Results
LLMs generally achieved high recall but low specificity, while valid papers ranged from 11% to 58% and correct URLs from 0% to 44%.
Takeaways & Limitations
Current LLMs should be used with caution for causal verification because they struggle to reject unsupported claims and reliably reference real scientific evidence.
Takeaways & Limitations
The study is preliminary and small-scale, evaluating only 17 hypotheses.
Abstract
from arXiv · showhide
The growing use of large language models (LLMs) for search and information retrieval underscores the need to evaluate their reliability in high-stakes domains such as healthcare. Although LLMs can effectively answer questions about diseases, symptoms, and treatments, their ability to accurately assess causal relationships and ground their conclusions in verified scientific evidence remains unclear. Here, we present a preliminary, small-scale study that investigates the accuracy of LLMs in evaluating causal medical claims and supporting them with peer-reviewed research. We propose an evaluation framework for causal hypothesis verification that can be used to systematically track the performance of existing and future LLMs. We assess the performance of eight LLMs on 17 medical causal hypotheses to evaluate whether they can reliably verify these hypotheses using scientific evidence from the literature. We systematically annotate the scientific evidence they provide according to six criteria (a total of 1,067 annotation points) and assess them with nine evaluation metrics. Our analysis shows that while LLMs exhibit strong recall, they often perform poorly at providing valid scientific articles and evidence for support and at rejecting unsupported hypotheses. These findings highlight a critical limitation of current LLMs, as they cannot yet be trusted fully to verify causal relationships from the biomedical literature. This work underscores the need for rigorous evaluation before using LLMs for search and retrieval in healthcare settings.
1 Introduction
Causal hypothesis verification is a promising but high-stakes use of LLMs because medical claims require reliable evidence. This study addresses an underexplored gap by evaluating whether LLMs can verify causal claims and retrieve supporting medical literature.
- Motivation: Causal reasoning supports intervention decisions, but medical applications require careful assessment of LLM limitations.The paper highlights healthcare as a high-stakes setting for using models on unstructured causal claims.
- Research gap: Prior research examined LLM causal reasoning and scientific-claim verification, but not retrieval of evidence specifically for causal medical claims.The study positions causal evidence retrieval as distinct from general causal reasoning and broader scientific-claim verification.
- Study aim: The study evaluates whether LLMs can answer medical causal queries and find literature supporting their conclusions.The target task includes verifying hypotheses such as whether diabetes causes renal disease and providing up to three scientific articles when supported.
- Contribution: The proposed framework systematically tracks causal-hypothesis-verification performance across existing and future LLMs.The evaluation uses 17 clinician-validated hypotheses and 1,067 annotation points despite the study’s small scale.
2 Evaluation Setup for Causal Hypothesis Verification
The evaluation pipeline collects structured LLM responses to medical causal hypotheses and uses human annotation to assess their accuracy and evidence quality. It combines standardized prompting, annotation procedures, and multiple performance metrics.
- Pipeline: A structured prompt collects and parses LLM responses for evaluating evidence supplied for causal relationships.The pipeline is designed to assess both the responses and their supporting scientific evidence.
- Prompting: The study uses a standardized prompt template across all LLMs and hypotheses.The same template is presented as the basis for consistent response collection.
- Annotation: Two annotators independently label every response, reconcile disagreements, and produce a final dataset of 1,067 annotation points.The annotations cover whether each provided paper is real and five criteria for each real paper.
- Metrics: Performance is measured with precision, recall, F1 score, specificity, and accuracy for causal-hypothesis classification.The metrics distinguish correct positive classifications, recovered true relationships, rejected false hypotheses, and overall classification performance.
3 Results
Across causal hypothesis classification and evidence grounding, LLMs showed stronger recall than specificity but struggled to provide valid, verifiable scientific support. Retrieval, model size, architecture, and reasoning style were associated with differing strengths across metrics.
- Causal hypothesis classification performance (RQ1): Accuracy ranged from 0.53 to 0.82 and F1 from 0.64 to 0.88 across causal hypothesis classification.
- Causal hypothesis classification performance (RQ1): Recall reached 1.0 for most models, while specificity remained between 0.17 and 0.50, indicating frequent false-positive predictions.
- Evidence accuracy and scientific grounding (RQ2): Valid papers ranged from 11% to 58% and correct URLs from 0% to 44%, while exact abstracts and quotations were generally inaccurate.Llama-4-Maverick reached 17% abstract accuracy and GPT-4o reached 41% quotation accuracy.
- Evidence accuracy and scientific grounding (RQ2): Models produced similar numbers of total papers and URLs, but valid evidence varied substantially; Gemini Pro and DeepSeek-R1 supplied 21 and 18 supporting papers, respectively.Real papers numbered 26 for DeepSeek-R1 and 28 for Gemini Pro, while correct URLs numbered 11 and 21.
- LLM search process: DeepSeek-R1, ChatGPT, and Gemini Pro used more sophisticated reasoning, whereas Mistral, Qwen, and DeepSeek-V3 relied more on surface-level keyword searches.
- Model size effect on responses: Larger models generally performed better, but architecture, training data, and retrieval capabilities contributed as much to metric differences as parameter count.Gemini 2.5 Pro achieved 0.58 real papers and 0.44 correct URLs, while Llama-4-Maverick achieved F1 0.88 and specificity 0.50.
- RAG architecture effect on responses: Models using external retrieval generally produced more real papers, valid URLs, and accurate quotations, but Llama-4-Maverick achieved the highest F1 score of 0.88 and conclusion accuracy of 0.82 despite emphasizing internal reasoning.
4 Conclusion
This preliminary study evaluates whether LLMs can verify biomedical causal hypotheses and ground their conclusions in verifiable scientific evidence. The models sometimes identify true relationships but struggle to reject unsupported claims and reference traceable evidence, motivating larger-scale investigation and cautious use.
- The study evaluates LLMs’ ability to judge causal validity and ground biomedical conclusions in verifiable scientific evidence.
- LLMs sometimes identify true causal relationships but consistently struggle to reject unsupported claims and reference real, traceable scientific evidence.
- The findings indicate that current LLMs should be used cautiously and not relied upon autonomously for evidence-based biomedical reasoning.