From Knowledge Generation to Knowledge Verification: Examining the BioMedical Generative Capabilities of ChatGPT

• Figure 1 is a graphical abstract of the knowledge generation and verification tasks performed.Source: Figure 1 · p. 3

• LLMs can accelerate tasks, but their generated knowledge raises concerns about reliability and scientific authenticity.Source: §1 · p. 4
• The paper focuses on verifying the factuality of ChatGPT-generated biomedical associations using relevant ontologies and PubMed abstracts.Source: §1 · p. 4
• The authors previously introduced xFakeSci to distinguish fake or AI-generated articles from real publications with high precision.Source: §1 · p. 4

2. Introduction

• Fact-checking and knowledge verification are presented as safeguards against hallucination and misinformation in generative AI, especially in biomedical applications.Source: §2 · p. 4

2.1. Literature review

• Prior work spans misinformation research, automated fact-checking, verification of generated content, biomedical LLM applications, and news and multilingual settings.The review describes approaches including external-resource cross-checking, radiology-report checks, sentence-level factuality assessment, and retrieval-augmented generation [29] [31].Source: §2.1 · pp. 4–5

2.2. Goal and objectives

• The study aims to test ChatGPT’s generation of disease-centric biomedical terms and associations, ontology-based term verification, PubMed verification, and consistency across four ChatGPT models.Source: §2.2 · p. 5

• Task 1 checks generated biomedical terms against relevant ontologies; an unverified term is not necessarily invalid because ontology coverage may be incomplete.Source: §3 · p. 5
• Task 2 treats associations as binary relations between term categories, such as disease and gene terms.A pair in the disease-gene relation represents a gene associated with a specific disease.Source: §3 · pp. 5–6
• Task 3 compares associations generated from simulated abstracts across ChatGPT models with the original associations from Task 2.Source: §3 · p. 6

4.1. Data generation via prompt engineering

• Prompt engineering generated 5,000 disease-linked associations for each of genes, symptoms, and drugs, with ontology identifiers and structured JSON outputs.A one-shot prompt example requests ten disease-symptom associations and checks whether the response is valid JSON.Source: §4.1 · pp. 6, 8

4.2. Term correctness verification against biomedical ontologies

• Terms were checked against specialized ontologies for diseases, drugs, genetic knowledge, and symptoms, using ontology names and synonyms for semantic verification.The ontologies are DOID, ChEBI, GO, and SYMP, respectively.Source: §4.2 · pp. 9–10

4.3. Association reliability verification against biomedical literature

• Association reliability was assessed by searching contextual PubMed abstract datasets for co-occurrence of disease-drug, disease-gene, or disease-symptom pairs.Coverage is the number of PubMed abstracts containing an association; ontologies verify terms but cannot verify the association itself.Source: §4.3 · pp. 10–11

5.1. Task 1 – Verification of the correctness of biomedical terms

• Task 1 checked generated terms against DOID, ChEBI, SYMP, and GO, using ontology names and synonyms for literal and semantic matching.Source: §5.1 · p. 12, Table 2 · p. 14

5.2. Tasks 1.1 – Verification of disease terms

• Combined disease-name verification reached 98% for disease-drug and 99% for disease-symptom links; disease-gene terms had 88% literal and 97% semantic verification.Literal and semantic matching verified 93% and 87% of disease-drug terms, and 97% and 82% of disease-symptom terms; combined matching was omitted for disease-gene terms.Source: §5.2 · p. 12

5.2.1. Tasks 1.2 – Verification of non-disease terms

• Drug names reached 91% combined verification (90% literal; 90% synonym), symptoms 61% combined (49% literal; 25% additional semantic), and gene names and processes 80% and 97%.Symptom matching was notably lower; the authors note that symptom terms not identified by the ontology also showed improved association-link coverage in the literature search.Source: §5.2.1 · p. 13

5.3. Task 2: Verification of the reliability of biomedical association links

• Across three periods, disease-drug coverage was 86%, 88%, and 90%; disease-gene coverage 83%, 83%, and 89%; disease-symptom coverage 49%, 53%, and 62%; genetic process-gene coverage rose from 23% to 83% and 89%.Source: §5.3 · p. 16

5.4. Task 3: Verification of the association consistency against ChatGPT-simulated Abstracts by various models

• In simulated abstracts, coverage was 1%–15% for disease-drug, 1%–4% for disease-gene, and 2%–29% for disease-symptom associations.Approximately 5,000 abstracts were generated per model; symptom associations had the highest match rates in this evaluation, contrasting with their lower performance against biomedical literature, and require further validation.Source: §5.4 · pp. 18–20, Table 5 · p. 19

6. Discussion

• The study uses prompt engineering to generate disease-centered associations linking diseases with symptoms, drugs, and genetic information, then checks them against ontologies and PubMed abstracts.It treats each association as a building block for more complex biomedical models.Source: §6 · pp. 20–21

6.1. Term correctness verification

• Symptom matching was hindered by lay language, verbosity, combined symptoms, punctuation inconsistencies, and missing ontology synonyms.Examples include “itchy blisters” versus “blister” and “rapid weight loss” versus “weight loss.”Source: §6.1 · p. 21

6.2. Association reliability and consistecny verification

• Some disease–drug and disease–gene associations had very high literature coverage, including diabetes mellitus–insulin and breast cancer–ATM.The cited examples had 3000+ and 9000 co-occurrences, respectively.Source: §6.2 · p. 21

6.3. Association consistency verification

• Single-abstract hits may be coincidental, although investigation found non-small-cell lung cancer–lapatinib correct and linked it to drug-repurposing evidence.The passage also notes lapatinib’s use for breast cancer with capecitabine.Source: §6.3 · pp. 21–22
• Hypertensive heart disease–losartan had weak literature coverage, despite published treatment evidence from Xu et al. and Suzuki et al..The publications appeared during 2009–2015; the authors hypothesize ChatGPT may have drawn on another source.Source: §6.3 · p. 22
• Associations from 2020–2024 publications had more coverage than older ones, with coverage significantly lower for publications at least a decade old.The generated associations in the losartan examples were found only in older literature.Source: §6.3 · p. 22
• Associations appeared in independently generated, generic simulated abstracts across models, though overlap with the original associations was low.The authors interpret this as evidence that generation was grounded rather than random.Source: §6.3 · p. 22

Symptom verification

• An LLM-driven semantic-similarity approach and self-consistency checks within or across models are proposed to improve symptom verification and disease–symptom associations.The authors also propose social-media data on adverse drug reactions and side effects as a possible extension.Source: §7 · p. 22

Models and association generation

• The authors plan to test open-source models to address cost limitations and improve unsupervised prompt-engineered association generation using biomedical literature.Proposed models include Phi-3, Llama, QWen2.5, and TiniZero.Source: §7 · p. 22
• Retrieval-augmented generation could supply relevant publications and use one document at a time for in-context prompt engineering aimed at factuality and explainability.The authors expect these approaches to increase confidence in generated knowledge and verification outcomes.Source: §7 · pp. 22–23

Exploring identified associations

• The authors propose using verified associations and PubMed abstracts to prompt exploration of related facts, then verifying outputs and building literature-driven heterogeneous mirror networks.The proposed exploration includes drug targets, side effects, alternatives, mechanisms, interactions, combinations, and repurposing evidence.Source: §7 · p. 23

Declaration of generative AI and AI-assisted

• ChatGPT generated study associations and simulated articles and assisted with LaTeX formatting; the authors wrote the paper without generative-AI writing assistance.Source: §8 · p. 23