Source-linked AI summary
Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?
Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, Xin Luna Dong
TL;DR
The paper asks how much factual knowledge LLMs confidently internalize and whether their factuality varies across knowledge of different popularity levels. It constructs Head-to-Tail and an automated evaluation framework, then evaluates 16 LLMs, finding notable limitations particularly for torso and tail entities.
Problem
The paper investigates LLM factuality, hallucinations, and whether LLMs might replace knowledge graphs, given limited knowledge of nuanced domain-specific details.
Method
The authors construct Head-to-Tail, an 18K-QA benchmark sampled across head, torso, and tail popularity buckets, and use automated factuality metrics.
Results
Evaluation of 16 LLMs shows consistently declining accuracy from head to torso to tail entities, with notable limitations especially for torso and tail facts.
Takeaways & Limitations
The findings suggest that improving LLM factuality requires approaches beyond standard model-size increases and instruction tuning.
Takeaways & Limitations
The study primarily evaluates least-ambiguous question forms and does not assess robustness to paraphrasing or diverse query formats.
Abstract
from arXiv · showhide
Since the recent prosperity of Large Language Models (LLMs), there have been interleaved discussions regarding how to reduce hallucinations from LLM responses, how to increase the factuality of LLMs, and whether Knowledge Graphs (KGs), which store the world knowledge in a symbolic form, will be replaced with LLMs. In this paper, we try to answer these questions from a new angle: How knowledgeable are LLMs? To answer this question, we constructed Head-to-Tail, a benchmark that consists of 18K question-answer (QA) pairs regarding head, torso, and tail facts in terms of popularity. We designed an automated evaluation method and a set of metrics that closely approximate the knowledge an LLM confidently internalizes. Through a comprehensive evaluation of 16 publicly available LLMs, we show that existing LLMs are still far from being perfect in terms of their grasp of factual knowledge, especially for facts of torso-to-tail entities.
1 Introduction
The paper frames LLM factuality and the possible relationship between LLMs and knowledge graphs through a benchmark covering facts across popularity levels. It introduces automated evaluation and finds uneven factual knowledge, especially for less-popular entities.
- Findings: GPT-4 accuracy decreases from head to torso to tail entities and averages only 31% on the benchmark.Even among the top-0.5% most popular entities in popular domains, evaluated LLMs achieve at best about 60% accuracy.
- Evaluation: The evaluation uses simple questions, concise answers, and an “unsure” option to approximate confidently internalized knowledge.The methodology distinguishes hallucinations from missing answers using automated metrics.
- Benchmark: Head-to-Tail is an 18K-question benchmark covering head, torso, and tail facts across domains and relationships.Entities and relationships are bucketed by popularity, with random sampling from each bucket.
- Research questions: The benchmark targets questions about how reliably LLMs answer factual queries and whether performance varies across head, torso, and tail facts.It also examines whether model size and instruction tuning improve factual knowledge.
- Contributions: The paper presents Head-to-Tail, an automated factuality evaluation methodology, and a comprehensive evaluation of 16 publicly available LLMs.The benchmark assesses factual knowledge across head, torso, and tail portions of knowledge graphs.
2 The Head-to-Tail Benchmark
Head-to-Tail is an 18K-question benchmark designed to evaluate LLM knowledge across popularity-based entity buckets and broad domains. Its evaluation uses concise answers, explicit “unsure” responses, and automated metrics to estimate factual knowledge while reducing unnecessary hallucinations.
- Benchmark construction: The benchmark uses DBpedia plus Movie, Book, and Academics data sources, while filtering selected domains to avoid questions requiring recent knowledge.The specific-domain sources are IMDb, Goodreads, MAG, and DBLP; their cutoff years precede the LLM training-data cutoff.
- Benchmark construction: Head, torso, and tail entities are partitioned using cumulative popularity, with separate partitions determined for different entity types.Popularity is approximated by traffic when available and by density, such as the number of facts or authored works, otherwise.
- Benchmark construction: 18K QA pairs span multiple domains and relationships, with entities sampled equally across head, torso, and tail popularity buckets.Specific domains contribute about 1K questions per bucket, while DBpedia contributes about 3K per bucket.
- Metrics: Accuracy, hallucination rate, and missing rate measure correct, incorrect or partially incorrect, and unanswered responses, respectively, and sum to 100%.LLM-based accuracy and hallucination metrics are primary because they achieved 98% agreement with manual judgments on 840 sampled answers.
- Evaluation methodology: The evaluation prompts LLMs for concise answers and “unsure” when uncertain, making rule-based scoring more reflective of factual correctness and reducing hallucinations.The benchmark combines simple question forms, concise responses, and uncertainty admission to approximate confidently internalized knowledge.
3 Experimental Analysis
Across 16 representative LLMs, factual accuracy declines from head to torso to tail entities, while overall performance and domain-level results remain limited. Model size and instruction tuning do not automatically improve factual knowledge, and evaluation choices affect hallucination and answer stability.
- 3.1 Models and configurations: 31% is the best overall accuracy achieved by GPT-4 on Head-to-Tail.GPT-4, ChatGPT, Llama 2-70B, and LLaMA-33B perform best on most metrics, but GPT-4 reaches only 31% overall accuracy.
- 3.2 RQ1: How reliable are LLMs in answering factual questions?: Accuracy consistently decreases from head to torso to tail entities across evaluated LLMs.The pattern is reported for GPT-4 and Llama 2-70B and observed for the other evaluated models as well.
- 3.2 RQ1: How reliable are LLMs in answering factual questions?: 46% for GPT-4 and 19% for Llama 2-70B are the accuracies on the top-10% most popular head questions.These figures are only slightly higher than performance on all head entities, and the missing rate remains notable.
- 3.3 RQ2: Do LLMs perform equally well on head, torso, and tail facts?: Head-to-tail ordering is not consistently associated with predicate-level performance across LLMs.The reported accuracy differences are not very high; predicate semantics and facts about head entities may contribute to this pattern.
- 3.4 RQ3: Does normal methods that improve LLMs increase the factuality?: LLaMA-33B modestly outperforms LLaMA-65B across subsets, with +0.4% in ALM and −1.9% in HLM on average.The comparison suggests that increased model size does not automatically translate to a better grasp of factual knowledge.
- 3.4 RQ3: Does normal methods that improve LLMs increase the factuality?: Instruction-tuned Vicuna and Falcon-Instruct have lower accuracy than their LLaMA and Falcon counterparts but produce “unsure” more often and still hallucinate frequently.For example, Vicuna-13B has a missing rate 26.9% higher than LLaMA-13B.
- 3.5 Robustness of our evaluation methodology: 1% is ChatGPT’s answer-regeneration disagreement rate when brief answers and “unsure” responses are both required.The disagreement rate falls from 18% without these requirements to 4% with brief answers, then to 1% when “unsure” examples are added; removing “unsure” raises hallucinations by 13 percentage points.
4 Discussions
The paper argues that factual knowledge should be represented through complementary symbolic and neural forms, while distinguishing research priorities by knowledge popularity and acknowledging evaluation boundaries.
- Long-tail knowledge is sparse in knowledge graphs and even more deficient in LLMs.
- Dual Neural KGs combine explicit triples for human understanding and explainability with implicit embeddings for machine comprehension and conversations.
- Head knowledge: Head knowledge remains a target for model training or fine-tuning because current LLM QA accuracy for popular entities is mediocre.
- Torso-to-tail and recent knowledge: Torso-to-tail and recent knowledge may be better represented as triples, requiring external-knowledge decisions, retrieval, and answer integration.
- The paper does not evaluate taxonomy capture and does not fully assess robustness to paraphrasing or diverse query forms.It selected the least ambiguous factual-question formulation, while supplementary results suggested question-form variation did not significantly affect evaluation results.
5 Related Work
Related work has evaluated LLM factuality and popularity effects, but Head-to-Tail systematically measures head, torso, and tail knowledge across broader sources, domains, and relations.
- Benchmarks: Head-to-Tail is the first benchmark specifically assessing LLM incorporation of head, torso, and tail factual information.
- LLM Evaluation: Prior factuality studies commonly focus on a single knowledge source and have not systematically evaluated head, torso, and tail entities.
- LLM Evaluation: Three concurrent works also connect language-model QA accuracy with fact popularity, but this work focuses on how much knowledge LLMs possess.
- LLM Evaluation: The benchmark spans different knowledge sources, domains, and rich relations, providing more quantified answers to RQ1–RQ3 than those works.
6 Conclusion
The paper introduces Head-to-Tail and an automated factuality-evaluation methodology to assess whether LLMs internalize facts across popularity levels. Results show notable limitations, especially for torso and tail entities, motivating blended symbolic-neural knowledge representations.
- Head-to-Tail is designed to assess LLMs’ ability to internalize head, torso, and tail facts.
- The paper presents automated evaluation metrics for measuring LLM factuality alongside the benchmark.
- Even the most advanced evaluated LLMs have notable limitations in representing factual knowledge, particularly for torso and tail entities.
- The paper suggests research on blending symbolic and neural forms of knowledge seamlessly.
A.1 List of Prompts
The appendix lists prompt templates for generating questions, answering them under several prompting settings, and automatically checking prediction correctness against ground-truth answers.
- Prompt 1: Question template drafting: Question-template drafting converts relation examples into templates with one blank for generating questions about the target object.
- Prompt 2: Correctness checking: Correctness checking asks an evaluator to label predictions as correct or incorrect using the question and ground-truth answers.
- Prompt 3: Question answering (Few-shot): Few-shot question answering requests concise answers and allows the model to say “unsure” when it does not know.
- Prompt 4: Question answering (Zero-shot): Zero-shot question answering removes the provided examples while retaining concise-answer instructions and the “unsure” option.
- Prompt 5: Question answering (In-domain): In-domain question answering uses an in-domain instance described in Section 3.5 while asking for concise answers.
- Prompt 6: Question answering (simply asking for concise answers): The concise-answer prompt requests a best guess when the model does not know, unlike prompts that instruct it to answer “unsure.”
A.2 Popularity measure in head-to-tail partition
Head-to-Tail measures popularity using domain-specific traffic or density signals. These signals quantify an entity’s exposure or representation within each source.
- IMDb popularity is measured by the number of votes received by a title.For people, the measure sums votes for titles they are known for, regardless of vote quality.
- Goodreads popularity is measured by the number of ratings received by a book.The rating value itself is not considered.
- MAG popularity is measured by the citation count of a scholarly article, conference, or journal.
- DBLP density is measured by the number of works authored by a scholar.
- DBpedia density is measured by the number of relational triples containing an entity.
A.3 Implementation details
The study evaluates ChatGPT and GPT-4 through the OpenAI API and runs other LLMs with specified model versions and reduced-precision formats on A100 GPUs.
- ChatGPT and GPT-4 were accessed through the OpenAI API using gpt-3.5-turbo-0301 and gpt-4-0613, respectively.
- The other models were run with Transformers on A100 80GB GPUs using float16 or bfloat16 formats.The study used original versions of LLaMA, Llama 2, Flan-T5, Falcon, Falcon-Instruct, RWKV v4 Raven, and Vicuna v1.1.
A.4 Impact of less naturally occurring questions
Additional analyses show that question formulation produces similar results, while domain aggregation and prompting affect missing-rate estimates without eliminating the head-to-tail pattern.
- The benchmark includes all predicates that support reasonable factual questions, while Table 10 focuses on predicates users are more likely to ask about.
- Movie and Book domains generally show higher performance, but accuracy remains fairly low and head-to-tail patterns persist.
- Cloze-style queries produce factuality results very similar to simple-formed questions for ChatGPT in the Movie domain.
- Over 30% of GPT-4’s top-10% head-bucket questions have missing answers when results are aggregated across domains.The aggregate includes the tail domain Academics.
- 23% is GPT-4’s missing rate for top-10% head entities in Movie, lower than the aggregate rate across studied domains.
- 15% is GPT-4’s Movie-domain missing rate when prompted for a best guess, but the hallucination rate increases.GPT-4 still answers “unsure” for 15% of questions after this prompting change.