Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

Motivation

• LLMs reduce knowledge-base construction costs and support flexible, context-aware reasoning, but may generate non-factual or misleading content.These capabilities motivate the survey’s focus on factual reliability.Source: §1 · p. 2
• Factual errors can influence decisions and cause harm in high-stakes domains such as health, law, finance, and autonomous driving.LLM outputs increasingly affect search, chatbots, content generation, and agent-based planning.Source: §1 · p. 2

Research gap and novelty

• The survey claims to be the first comprehensive study of LLM factuality, addressing domain-specific factuality and outdated information overlooked by prior surveys.It distinguishes factuality issues from hallucination issues while broadening coverage beyond domain specialization.Source: §1 · pp. 2–4

Survey scope and organization

• The survey covers factuality’s definition and impact, evaluation, underlying mechanisms and error causes, and enhancement approaches across standalone and Retrieval-Augmented LLMs.Its structure is illustrated in Figure 1 and includes quantitative assessment alongside analysis of factuality mechanisms.Source: §1 · p. 4

2.1 Large Language Models

• The survey primarily considers decoder-only generative pretrained LLMs with emergent abilities, while also including selected encoder-decoder models and excluding encoder-only systems.Examples include ChatGPT, LLaMA [254] [255], and T5 [213].Source: §2.1 · p. 4

2.2 Factuality

• Factuality concerns whether LLM-generated content follows reliable, undisputed facts across commonsense, world knowledge, and domain knowledge.The survey considers only cases with clear meaning and determinable truth, excluding minor errors in reliable sources.Source: §2.2 · p. 4
• Factuality overlaps with hallucination, outdated information, and domain specificity, but focuses specifically on learning, acquiring, recalling, and using factual knowledge.Hallucinations may be factually accurate but misaligned with prompts, outdated information reflects superseded knowledge, and domain specificity requires specialized knowledge.Source: §2.2 · pp. 5–6
• The survey covers standard LLM answering and chatting alongside retrieval-augmented generation, which must both generate accurate responses and select pertinent snippets from retrieved sources.Retrieval is highlighted as a prevalent approach for improving factuality.Source: §2.2 · p. 7

2.3 Impact

• Factuality failures reduce LLM usability and can cause societal, economic, and legal harms, including fabricated case law, defamatory claims, and court-mandated disclosure of AI-generated material.Reported incidents include sanctions against a lawyer and false allegations generated about a law professor and an Australian mayor [48].Source: §2.3 · p. 7
• Real-world healthcare deployment remains risky because focused benchmarks may not capture instability, non-robust personalized recommendations, erroneous rankings, omissions, and other factual errors.The survey notes that evaluating LLMs for real-world decision-making remains challenging despite promise on medical datasets.Source: §2.3 · pp. 7–8

3 FACTUALITY EVALUATION

• Factuality evaluation is presented as central to the reliability and trustworthiness of LLM-generated content across applications.Source: §3 · p. 8

3.1 Factuality Evaluation Metrics

• The metric taxonomy comprises rule-based, neural, human, and LLM-based evaluation methods.The survey lists these metrics in Table 3.Source: §3.1 · pp. 8–9

3.1.1 Rule-based evaluation metrics.

• Rule-based metrics dominate factuality assessment because they are consistent, predictable, easy to implement, and reproducible, but they may miss contextual nuance and naturalness.Common examples include Exact Match, Accuracy, Precision, Recall, AUC, F-Measure, calibration, Brier score, MC1, MC2, BLEU, ROUGE, METEOR, and QUIP-Score.Source: §3.1.1 · pp. 9–10

3.1.2 Neural Evaluation Metrics.

• Neural metrics such as ADEM, BERTScore, BLEURT, and BARTScore learn or exploit semantic and lexical similarity to evaluate generated text against references.BARTScore uses pre-trained sequence-to-sequence models and has outperformed existing top-scoring metrics in most tested settings across datasets and evaluation perspectives.Source: §3.1.2 · pp. 10–11
• Human evaluation captures contextual and linguistic nuance that automation may miss, while automated evaluation provides scalability and consistency, motivating hybrid frameworks.Human judgments remain vulnerable to subjectivity, inconsistency, and error, whereas automated methods support quantitative model comparisons.Source: §3.1.3 · p. 11

3.1.3 Human Evaluation Metrics.

• FActScore decomposes long-form outputs into atomic facts and found factual precision of 42%–71% for biographies generated by several commercial LLMs, decreasing for rarer entities.The score is the percentage of atomic facts supported by a reliable knowledge source; evaluated systems included InstructGPT, ChatGPT, and retrieval-augmented PerplexityAI.Source: §3.1.3 · pp. 11–12

3.1.4 LLM-based Metrics.

• LLM-based evaluators improve evaluation efficiency and flexibility, but their validity depends on appropriate model and decoding-strategy choices and may remain limited in domain coverage.GPTScore evaluates 22 aspects across 37 datasets, while LLM-Eval showed effectiveness, efficiency, and adaptability across multiple benchmark datasets; TruthfulQA’s truth * info metric combines truthfulness with informativeness.Source: §3.1.4 · pp. 12–13

General factuality benchmarks

• MMLU [93] evaluates multitask accuracy across 57 subjects, while TruthfulQA [147] tests whether models avoid human-like misconceptions across 817 questions in 38 categories.MMLU measures world knowledge and problem-solving, whereas TruthfulQA targets truthfulness in generated answers; recent models remain below expert-level performance across MMLU tasks [193].Source: §3.2 · p. 13

Hallucination evaluation

• HaluEval [137] evaluates hallucination recognition using generated and human-annotated samples produced through a ChatGPT-based sampling-then-filtering framework.It targets content that conflicts with sources or cannot be verified from factual knowledge, while characterizing hallucination types and prevalence.Source: §3.2 · p. 13

Domain-specific evaluation

• On C-Eval, only GPT-4 achieved average accuracy above 60%, while the evaluation assessed advanced language-model abilities within a Chinese context, including culture, history, and laws.GPT-4, ChatGPT, and Claude were the top performers despite not being exclusively tailored for Chinese data.Source: §3.2 · pp. 15–16

Self-knowledge evaluation

• GPT-4 achieved the highest SelfAware F1 score of 75.5 versus 85.0 for humans on answerable and unanswerable questions.The dataset contains 2,858 unanswerable and 2,337 answerable questions; larger models and in-context learning improve performance.Source: §3.2 · p. 16

Dynamic factuality evaluation

• REALTIMEQA evaluates weekly questions about current events, finding that GPT-3 can update answers from newly retrieved documents but sometimes remains outdated when evidence is insufficient.The platform addresses instantaneous information needs that static open-domain question-answering datasets do not capture.Source: §3.2 · p. 16
• FreshQA [261] tests stable, fast-changing, and false-premise questions, while FRESHPROMPT significantly improves performance by integrating real-time search data.Its RELAXED and STRICT human-evaluation modes show that outdated training data weakens LLM performance and motivate continual refreshing.Source: §3.2 · p. 16

Evaluation methodologies

• Manakul et al. [168] evaluate factuality through sentence-level annotation, passage-level aggregation, and precision-recall curves distinguishing non-factual, non-factual* and factual sentences.Synthetic Wikipedia articles generated with GPT-3 are manually labeled as major inaccurate, minor inaccurate, or accurate; Cohen’s κ assesses annotator agreement and score distributions identify total inaccuracies.Source: §3.3 · p. 17
• BSChecker detects factual errors by extracting subject-predicate-object claims and classifying them as entailment, contradiction, or neutral across zero-, noisy-, and accurate-context settings.Its settings cover closed-book QA, retrieval-augmented generation, summarization, closed-QA, and information extraction.Source: §3.3 · p. 17

Adversarial evaluation

• A false-premise evaluation found that GPT-3.5 answered all 50 adversarial questions incorrectly, motivating paired true-premise questions and strict whole-answer correctness.The paired design evaluates performance on both false and true premises rather than accepting partially correct answers.Source: §3.3 · p. 17

Comparative chatbot evaluation

• GPT-4 achieved the highest weighted claim-level and response-level factual accuracy among evaluated chatbots, while supervised fine-tuned models declined on math, code, and scientific prompts.FACTOOL evaluated GPT-4, ChatGPT, Claude, Bard, and Vicuna across knowledge-based QA, code, math, and scientific prompts using claim- and response-level accuracy.Source: §3.3 · p. 19

LLM-based evaluation

• LLMs performed well but imperfectly on open-question answering, while GPT-3.5 and direct LLM factuality scores showed poor agreement with human correctness judgments.GPT-3.5 failed to reliably assess generated responses even when given golden answers, and GPT-2/GPT-4 factuality scores for summaries had no significant correlation with human evaluations.Source: §3.3 · p. 19

Retrieval-augmented evaluation

• RAGAS [61] evaluates faithfulness, answer relevance, and context relevance without references, and aligned more closely with human judgments than GPTScore and GPT Ranking.Its WikiEval dataset contains question-context-answer triplets with human judgments; faithfulness is computed from supported assertions, while relevance measures use generated questions, embeddings, and retrieved-sentence proportions.Source: §3.3 · p. 20

Overview

• Table 7 organizes domain-specific factuality benchmarks by domain, tasks, datasets, and evaluated LLMs.These resources support both evaluation of LLM capabilities and advancement of specialized applications.Source: §3.4 · p. 20, Table 7 · p. 21

Finance

• FLARE evaluates financial LLMs across NLP and prediction tasks, comparing FinMA with BloombergGPT and other LLMs under zero-shot or few-shot settings.Its subtasks include sentiment analysis, headline classification, named entity recognition, question answering, and stock movement prediction, with task-specific metrics.Source: §3.4 · pp. 21–22

E-commerce and oceanography

• EcomInstruct tests EcomGPT on 12 e-commerce tasks emphasizing cross-language and cross-task generalization, while OceanBench evaluates LLMs on 15 expert-verified oceanography task types.EcomInstruct uses generation, classification, and text-generation metrics; OceanBench samples are automatically generated from seed data and manually verified by experts.Source: §3.4 · pp. 22–23

Medicine

• CMB evaluates Chinese medical knowledge and clinical consultation across hundreds of thousands of questions, while Huatuo-26M provides over 26 million Chinese medical QA pairs.CMB covers all clinical specialties and multiple professional levels; Huatuo-26M supports medical QA, classification, prediction, and treatment-recommendation research, though consultation-record text is not publicly available.Source: §3.4 · pp. 22–23

Biomedicine

• GeneTuring evaluates GeneGPT on nine NCBI-related tasks spanning nomenclature, genomic location, functional analysis, and sequence alignment against general, biomedical, and retrieval-augmented LLMs.GeneGPT is assessed in full and slim prompt settings, with its results automatically evaluated while comparison results are manually evaluated from the original benchmark.Source: §3.4 · p. 23

Law

• LegalBench covers 162 tasks across six legal-reasoning types, whereas LawBench tests 51 LLMs on 20 tasks involving legal memorization, understanding, and application.LawBench reported GPT-4 as the top-performing model and found that legal fine-tuning improved results, while reliable legal performance remained challenging.Source: §3.4 · p. 23

4.1.1 Knowledge Storage.

• Factual knowledge is primarily encoded in early-layer feed-forward networks, whose memory-like keys, values, and activations represent individual facts, although their broader organization remains unclear [173] [83].Value vectors often capture human-interpretable concepts, while language-independent and degenerate neurons encode multilingual and redundant information [79] [80] [30].Source: §4.1.1 · p. 24

4.1.2 Knowledge Completeness and Awareness.

• LLMs misjudge their factual knowledge boundaries and are often overconfident, while retrieval, instructions, and in-context learning can partly improve knowledge-gap detection [217] [57].GPT-4 performs better than other models on the SelfAware dataset, larger models calibrate better, RLHF can impair calibration, and temperature adjustment can correct that issue [120].Source: §4.1.2 · pp. 24–25

4.1.3 Contextual Influence and Knowledge Conflict.

• LLMs balance parametric knowledge against retrieved context imperfectly: larger models may favor internal knowledge, noise reduces contextual use, and coherent counterfactual evidence can improve faithfulness [135] [317] [25].Models also struggle to assess alignment with references and remain sensitive to irrelevant information, although self-consistency and exemplar design can improve distractor resistance [295].Source: §4.1.3 · pp. 25–26
• Knowledge conflicts increase uncertainty, encourage reliance on parametric knowledge when evidence is irrelevant, and reveal differing behavior across FiD, RAG, and newer LLMs [159] [24] [279].FiD memorizes less than 3.6% of NQ answers, whereas newer LLMs may accept coherent contradictory evidence but show confirmation bias; conflict-detection tasks seek user agency [270].Source: §4.1.3 · pp. 26–27

4.2.1 Model-level Causes.

• Factual errors arise from model-level deficits including missing domain knowledge, outdated or forgotten information, incomplete memorization, and failures to reason over known facts [161] [84] [272] [298] [165].These causes are organized alongside retrieval- and inference-level factors, with Table 1 illustrating representative errors.Source: §4.2 · p. 27, §4.2.1 · p. 27

4.2.2 Retrieval-level Causes.

• Retrieval-level errors result from insufficient, misleading, distracting, or misinterpreted evidence, while long contexts make information in the middle harder to use [141] [159] [197] [163] [154] [289].Instruction tuning and improved retrieval can reduce susceptibility to distracting or misinterpreted information [163] [289].Source: §4.2.2 · p. 28

4.2.3 Inference-level Causes.

• Inference-level errors include snowballing from early mistakes, decoding failures, and exposure bias that favors frequent or overrepresented patterns over factual or relevant outputs [258] [301] [35] [169] [67] [72].Exposure bias contributes to poorer performance on underrepresented cases, including non-binary or neutral gender identification [96].Source: §4.2.3 · pp. 28–29

Standalone LLMs

• Standalone LLM methods enhance factuality during LLM generation and inference phases.Source: §5 · p. 29

Retrieval Augmented Generation

• Retrieval-augmented generation uses retrieval-phase strategies to enhance LLM factuality.Source: §5 · p. 29

Domain Factuality Enhancement

• Domain-specific LLM methods improve factual accuracy within their respective domains.Source: §5 · p. 29, Figure 2 · p. 29

• Table 8 summarizes enhancement methods and improvements over baseline LLMs, but results across studies may not be directly comparable because settings differ, including zero-shot, few-shot, and full settings.Source: §5 · p. 29

5.1.1 Pretraining-based.

• Pretraining-based methods improve factual knowledge by deduplicating corpora, prioritizing informative words with PMI, and structuring sentences with TOPICPREFIX and sentence completion loss [132].These approaches reduce memorized-text recall, improve factual recall and other closed-book tasks, and further reduce factual errors compared with standard factual-domain adaptive training.Source: §5.1.1 · pp. 30–31

5.1.2 Supervised Finetuning.

• Supervised finetuning injects labeled or structured knowledge, with KG-triplet training outperforming alternative knowledge-enhancement methods on closed-book QA exact match and contrastive learning matching SOTA KB-based methods with better efficiency and scalability.Model editing precisely updates factual representations, whereas weight-editing robustness and generalization remain open questions; ITI offers a less intrusive activation-editing alternative.Source: §5.1 · p. 29, §5.1.2 · pp. 31–32
• Inference-Time Intervention [139] improved Alpaca’s TruthfulQA truthfulness from 32.5% to 65.1%, while FactTune reduced errors by 50% in Llama biography generation and by 20+% in medical QA.FactTune uses reference-based or reference-free factuality scoring with Direct Preference Optimization [212]; ITI adjusts truth-related activation directions during inference.Source: §5.1.2 · p. 32

5.1.3 Multi-Agent.

• Multi-agent debate and interrogation improve factuality by having models compare, verify, and converge on answers, with LM-vs-LM fact-checking outperforming baseline F1 by ten to over twenty points across LAMA, PopQA, NaturalQA, and TriviaQA.Multi-agent debate can also be applied directly to existing black-box models using the same prompts.Source: §5.1.3 · pp. 32–33

5.1.4 Novel Prompt.

• Novel prompts elicit factual knowledge through generated contextual documents, grounding or anti-grounding instructions, task decomposition, and Chain-of-Verification’s draft-question-answer-revision loop.GENREAD outperformed Wikipedia retrieval on knowledge-intensive tasks; QUIP-Score prompting increased grounding scores without significantly improving traditional QA metrics; CoVe reduced errors across several tasks.Source: §5.1.4 · pp. 33–34

5.1.5 Decoding.

• Decoding methods improve factuality by reducing sampling randomness over sentence generation or contrasting later and earlier layers, while preserving generation quality or enhancing truthfulness across tasks.Factual-nucleus sampling dynamically decreases nucleus probability and uses an ω-bound to preserve diversity; Decoding by Contrasting Layers [35] leverages localized factual knowledge in later-layer logits.Source: §5.1.5 · p. 34

5.2 On Retrieval-Augmented Generation

• RAG enhancement is broadly categorized into normal retrieval, interactive retrieval, adaptation to RAG, and retrieval from additional parametric memories or knowledge graphs.These approaches respond to RAG’s central risks: insufficient information and misinterpretation of retrieved data.Source: §5.2 · pp. 34–35

5.2.1 Normal RAG Setting.

• Normal RAG loads and transforms documents, embeds and stores text, then retrieves relevant passages for generation, with retrieval databases also scaling language-model knowledge.Retro uses a database exceeding 5 trillion tokens and achieves Pile performance comparable to GPT-3 with 25× fewer parameters; retrieved evidence lets 7B Gopher surpass 280B Gopher by up to 30% on NQ.Source: §5.2.1 · p. 35

5.2.2 Interactive Retrieval.

• Interactive retrieval improves factual generation by dynamically deciding when and what to retrieve, while CoT and agent-based methods guide reasoning with external evidence.FLARE outperforms baselines across Multihop QA, Commonsense Reasoning, Long-form QA, and Open-domain Summarization; ReAct reduces incorrect reasoning or facts from 14% with CoT to 6% and reports no factual errors among its incorrect answers, though 23% arise from search errors.Source: §5.2.2 · p. 36

5.2.3 Retrieval Adaptation.

• Retrieval adaptation aligns LLM generation with retrieved content through prompting, supervised fine-tuning, or preference optimization, because retrieval alone may not improve factual answering.LLM-Augmenter reduces ChatGPT errors without sacrificing fluency or informativeness, while ATLAS achieves notable Natural Questions accuracy using 64 examples and 50× fewer parameters than a 540B model.Source: §5.2.3 · pp. 37–39

5.2.4 Retrieval on External Memory.

• External-memory methods store or retrieve textual, parametric, entity, or model-based knowledge to improve factuality, including memory-augmented layers, adapters, editing, and correction systems.PURR retrieves five web-page summaries, creates noisy samples, and trains a sequence-to-sequence editor to correct them, increasing F1 scores over EFEC.Source: §5.2.4 · pp. 39–40

5.2.5 Retrieval on Structured Knowledge Source.

• Structured-source retrieval uses query reformulation, specialized interfaces, or prompt injection to make knowledge graphs, tables, and databases usable for LLM reasoning.Knowledge-graph prompting improves knowledge-graph question answering by up to 48% on average over baselines without knowledge graphs, while StructGPT improves KGQA, TableQA, and Text-to-SQL in few-shot and zero-shot settings.Source: §5.2.5 · pp. 40–41

Motivation

• Domain-specific LLMs target knowledge deficits that constrain reliable application in specialized fields.The survey frames domain factuality enhancement as a concern for both academia and industry.Source: §5.3 · p. 42

Domain coverage

• Table 9 catalogs domain-factuality enhanced LLMs across healthcare/medicine, finance, law/legal, geoscience/environment, education, food testing, and home renovation.The table lists each domain-specific LLM together with its respective enhancement methods.Source: §5.3 · p. 43

Enhancement methods

• Continual pretraining updates general-purpose models with domain-specific data and evolving information to maintain accuracy and relevance in rapidly changing fields [287] [299].The approach repeatedly fine-tunes a pretrained model as new domain knowledge becomes available.Source: §5.3 · p. 43
• Continual SFT fine-tunes models on labeled domain data, improving accurate and contextually relevant responses when legal, medical, or financial annotations accumulate over time [10].It is especially useful when domain-specific labeled data are available continuously.Source: §5.3 · p. 43
• One enhancement route is training from scratch, which starts with minimal prior knowledge and can suit completely new domains or tasks with limited relevant data, but requires substantial computational resources and time.Source: §5.3 · p. 43

• Healthcare LLMs span clinical classification, consultation, medical question answering, de-identification, biomedical NLP, genomics, guideline recommendation, and molecular generation.Examples include CohortGPT [87], ChatDoctor, DeID-GPT [157], BioMedLM, BioGPT [164], GeneGPT [118], Almanac [296], and MolXPT [158].Source: §5.4 · pp. 43–44
• HuatuoGPT [299] achieves state-of-the-art medical-consultation performance by continually pre-training Baichuan-7B and Ziya-LLaMA-13B-Pretrain-v1 on ChatGPT-distilled and doctor data.Source: §5.4 · p. 44
• Zhongjing [287] enables complex Chinese medical dialogue and proactive inquiry through a comprehensive pipeline and 70,000 authentic doctor-patient dialogues in CMtMedQA.It uses Ziya-LLaMA-13B-v1 and is evaluated on CMtMedQA and huatuo-26M [138].Source: §5.4 · p. 44
• LLM-AMT improves GPT-3.5-Turbo and LLaMA-2-13B medical question answering by incorporating 51 MedQA textbooks and Wikipedia through a Hybrid Textbook Retriever.Source: §5.4 · p. 44
• Other systems enhance healthcare reliability with knowledge graphs, dialogue datasets, retrieval, external APIs, and specialized sources such as HIPAA identifiers, PubMed, NCBI, and clinical guidelines.DISC-MedLLM [10] uses knowledge graphs, reconstructed dialogues, and human-guided preference rephrasing; CohortGPT [87] uses knowledge-graph assistance and CoT sample selection; ChatDoctor uses 100,000 patient-doctor dialogues and self-directed retrieval; DeID-GPT [157] uses HIPAA identifiers; BioMedLM trains on PubMed; GeneGPT [118] uses NCBI Web APIs; Almanac [296] retrieves medical guidelines.Source: §5.4 · p. 44

Legal-domain capabilities

• Legal LLMs provide context-aware assistance for legal questions, document generation, and expert advice by leveraging extensive legal-text corpora.LawGPT [189] and ChatLaw [45] exemplify fine-tuned systems for comprehensive legal assistance.Source: §5.5 · pp. 44–45

Retrieval augmentation

• GPT-4 explanations of legal terms improved in quality when augmented with case-law retrieval compared with a baseline, addressing factual accuracy and hallucination.Savelka et al. [221] evaluated baseline and retrieval-augmented approaches using a legal information retrieval module.Source: §5.5 · p. 45

Domain adaptation

• Domain adaptation methods inject legal knowledge during continual training, design supervised fine-tuning tasks, and incorporate retrieval modules to improve factuality in generation.Huang et al. [104] released data and a model for further research on Chinese legal tasks.Source: §5.5 · p. 45

Chinese legal LLMs

• ChatLaw combines factuality-oriented data screening, self-attention error handling, and vector-database and keyword retrieval for Chinese legal tasks.Its fine-tuning data includes original legal materials, regulation- and judicial-interpretation-based data, and crawled legal consultations, using Ziya-LLaMA-13B as the backbone.Source: §5.5 · p. 45

5.6 Finance Domain-enhanced LLMs

• Finance domain-enhanced LLMs use business-focused datasets and fine-tuning to support financial text analysis, e-commerce instructions, sentiment identification, named entity recognition, and question answering.They are also designed for zero-shot generalization across diverse tasks and benchmarks.Source: §5.6 · p. 45
• EcomGPT [143] is a BLOOMZ-based model fine-tuned on EcomInstruct, a 2.5 million-instruction dataset covering product information, user reviews, and 12 e-commerce evaluation tasks.EcomInstruct defines atomic and Chain-of-Task tasks spanning classification, generation, extraction, and other e-commerce scenarios.Source: §5.6 · p. 45
• BloombergGPT [278] is a 50 billion-parameter financial model trained on 363 billion tokens, including 51.27% Bloomberg financial data, and evaluated on sentiment analysis, named entity recognition, binary classification, and question answering.It uses a decoder-only causal language model architecture based on BLOOM.Source: §5.6 · pp. 45–46

Geoscience and Environment

• K2 [52], trained on 5.5 billion geoscience tokens and GeoSignal, introduced GeoSignal and GeoBench for geoscience instruction tuning and evaluation.K2 is a LLaMA-7B continuously trained on a geoscience corpus and fine-tuned with GeoSignal.Source: §5.7 · p. 46
• HouYi [8] targets renewable energy using the 1.1-million-source REAP dataset and fine-tuning from general LLMs such as ChatGLM-6B.REAP contains over 1.1 million renewable-energy academic literature sources.Source: §5.7 · p. 46
• OceanGPT [14] applies automatically collected ocean-domain data through DoInstruct and achieves higher ocean-science knowledge expertise on OceanBench.It is presented as the first LLM specifically designed for the ocean domain.Source: §5.7 · p. 46

Education

• GrammarGPT [64] supports Chinese grammatical-error correction using ChatGPT-generated and human-annotated data, heuristic guidance, and the phoenix-inst-chat-7b backbone.It is an open-source LLM designed for native Chinese grammatical error correction.Source: §5.7 · p. 46

Food

• FoodGPT [209] addresses food-testing hallucinations through incremental pretraining on structured knowledge and scanned documents, plus a knowledge graph external knowledge base.Its backbone is Chinese-LLaMA2-13B.Source: §5.7 · p. 46

Home renovation

• ChatHome [277] combines domain-adaptive pretraining and instruction tuning on professional, standards, and web content, evaluated with C-Eval, CMMLU, and EvalHome.Its Baichuan-13B backbone is trained with sources including National Standards, Domain Books, Domain Websites, and WuDaoCorpora.Source: §5.7 · p. 46

Survey scope

• The survey covers LLM factuality from its definition and implications through evaluation, underlying mechanisms, and enhancements for standalone, retrieval-augmented, and domain-specific LLMs.Its evaluation coverage includes benchmarks, metrics, specific studies, and domain-specific assessments.Source: §6 · p. 47

Open challenges

• Factuality evaluation remains challenging because natural-language variability and nuance complicate reliable assessment.The survey identifies robust evaluation metrics as a key direction for future research.Source: §6 · p. 47
• How LLMs store, update, and produce facts remains insufficiently understood, motivating deeper investigation of their neural architectures.This unresolved understanding is presented as a central challenge despite the survey’s analysis of intrinsic factuality mechanisms.Source: §6 · p. 47

Future directions

• Continual training and retrieval show promise for improving factuality but remain limited, leaving enhancement techniques an open research priority.Future work is also needed to innovate on enhancement methods as LLMs become more integrated into digital systems.Source: §6 · p. 47