PaperlayerView exact revision on arXiv

Source-linked AI summary

Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts

Wenyu Huang, Guancheng Zhou, Mirella Lapata, Pavlos Vougiouklis, Sebastien Montella, Jeff Z. Pan

arXiv:2405.06524v1cs.CL

TL;DR

LLMs struggle with knowledge-intensive questions involving long-tail facts, and existing benchmarks can limit relation diversity. The paper introduces LTGen, a benchmark for evaluating LLMs across non-parametric knowledge settings; its results show benefits from external knowledge and, in most cases, stronger performance from KG triples than passage-based prompting.

  • Problem

    LLMs can struggle with knowledge-intensive generation involving long-tail entities, while template-based PopQA covers only a limited set of relations.

    Source: §1 · p. 1, §2 · p. 3

  • Method

    The paper proposes a fully automatic, template-free pipeline to build LTGen, a benchmark with question-answering and conversational QA tasks.

    Source: §1 · p. 1

  • Results

    Non-parametric knowledge improves LLM performance, especially for high-rarity or multi-triple questions, and KG-triple prompts are more effective than passages in the paper's reported comparison.

    Source: §1 · p. 2, §7 · p. 13

  • Takeaways & Limitations

    KG triples offer an effective and efficient source of non-parametric knowledge, while combining triples with passages can reduce hallucinations.

    Source: §1 · p. 2, §7 · p. 13

    Entity rarity in the benchmark is operationalized using Wikipedia pageviews, so its long-tail sampling follows that popularity measure.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Although Large Language Models (LLMs) are effective in performing various NLP tasks, they still struggle to handle tasks that require extensive, real-world knowledge, especially when dealing with long-tail facts (facts related to long-tail entities). This limitation highlights the need to supplement LLMs with non-parametric knowledge. To address this issue, we analysed the effects of different types of non-parametric knowledge, including textual passage and knowledge graphs (KGs). Since LLMs have probably seen the majority of factual question-answering datasets already, to facilitate our analysis, we proposed a fully automatic pipeline for creating a benchmark that requires knowledge of long-tail facts for answering the involved questions. Using this pipeline, we introduce the LTGen benchmark. We evaluate state-of-the-art LLMs in different knowledge settings using the proposed benchmark. Our experiments show that LLMs alone struggle with answering these questions, especially when the long-tail level is high or rich knowledge is required. Nonetheless, the performance of the same models improved significantly when they were prompted with non-parametric knowledge. We observed that, in most cases, prompting LLMs with KG triples surpasses passage-based prompting using a state-of-the-art retriever. In addition, while prompting LLMs with both KG triples and documents does not consistently improve knowledge coverage, it can dramatically reduce hallucinations in the generated content.

1. Introduction

LLMs can hallucinate on knowledge-intensive tasks involving long-tail entities, while existing long-tail benchmarks leave gaps in question diversity and structured knowledge evaluation. The paper introduces LTGen and uses it to compare LLMs across different knowledge settings.

1. Introduction

  • PopQA uses predefined templates, single entity-relation pairs, and unstructured knowledge, limiting relation diversity and structured-knowledge evaluation.The paper identifies these constraints as motivations for broader benchmark coverage.Source: §1 · p. 1
  • LTGen is a fully automatic, template-free benchmark with question-answering and conversational QA tasks, including multi-relation questions in LTGen-Conv.The paper frames its evaluation around how LLMs perform across factual knowledge types and how they benefit from structured and unstructured non-parametric knowledge.Source: §1 · pp. 1–2
  • LLMs perform poorly on long-tail questions, with performance declining as entity rarity and the number of required reference triples increase.Non-parametric knowledge improves performance, and combined KG-triple and passage prompting achieves the highest NLI-based metric scores across settings.Source: §1 · p. 2

2. Related Works

This section reviews structured knowledge bases and question answering over them, then discusses gaps in existing knowledge-intensive and long-tail benchmarks. It highlights limited long-tail coverage and template-constrained relation diversity.

Knowledge Bases (KBs)

  • Knowledge bases encode relational triples, literal attribute-value triples, and qualifiers that extend relational or literal triples with key-value information [11, 12].Entities, concepts, attributes, and relations are described as core knowledge-base elements.Source: §2 · p. 2

Question Answering Over Knowledge Bases (KBQA)

  • KBQA derives answers from structured knowledge bases; semantic parsing executes symbolic logical forms, while information retrieval gathers and ranks relevant entities and relations.The section also notes a shift from single-turn KBQA toward conversational settings.Source: §2 · p. 3

KIG Benchmarks

  • KIG benchmarks such as KILT contain few long-tail examples, and it remains unclear whether LLMs answer some benchmark questions from factual knowledge or memorized question-answer pairs.The paper argues that upgraded LLMs strengthen the need for challenging evaluation benchmarks.Source: §2 · p. 3

Long-tail benchmarks

  • PopQA's predefined templates cover a limited set of relations, constraining evaluation of diverse relation types users may ask about.Source: §2 · p. 3

3. LTGen: Long-Tail Generation Tasks

LTGen is an automatically generated benchmark with simple QA and multi-relation conversational QA. Its construction selects long-tail entities, retrieves and filters triples, generates samples, and includes a manual quality check.

3. LTGen: Long-Tail Generation Tasks

  • LTGen comprises LTGen-QA for simple question answering and LTGen-Conv for multi-relation, multi-turn conversational QA.The benchmark comparison is described as assessing completeness across several criteria.Source: §3 · p. 3

3.1. Benchmark Construction

  • The pipeline samples Wikidata entities and defines rarity by average monthly Wikipedia pageviews, following the assumption in [6].The three construction stages are long-tail entity selection, triple retrieval, and sample generation.Source: §3.1 · pp. 3–4
  • Triple retrieval excludes non-informative relations and entities appearing in fewer than 5 or more than 100 triples.The lower cutoff is intended to make questions more challenging; the upper cutoff reflects prompting efficiency.Source: §3.1 · p. 4
  • GPT-4 generates template-free entity-relation questions and dialogues from collected triples, enabling automatic annotation of dialogue turns against relevant gold triples.Source: §3.1 · pp. 4–5

3.2. Data Quality Checking

  • A manual check of 400 LTGen-Conv samples found an average annotation error rate of about 2.25%, or about 3% when both annotators had to agree on correctness.Source: §3.2 · p. 5

4. Non-parametric Memories Collection

The section gathers unstructured passages and structured KG triples as non-parametric knowledge, using retrieval and ranking strategies to select knowledge relevant to questions.

4.1. Unstructured Knowledge from Passages

  • The pipeline uses Contriever to retrieve unstructured knowledge from passages.Source: §4.1 · p. 5

4.2. Structured Knowledge from KGs

  • KG triples are more compact than passages and contain less noise, which may otherwise mislead LLMs.A triple usually takes fewer than 20 tokens to encode, compared with a 100-word passage.Source: §4.2 · p. 5

4.2.1. KG Triples Retrieval

  • KG retrieval follows a modular pipeline, with entity tagging, triple retrieval, and ranking for question relevance.The ranking stage explores LLM prompting and an AMR-based method.Source: §4.2.1 · pp. 5–6

4.2.2. AMR-Based Triples Ranking

  • AMR-based ranking predicts whether an entity–relation pair is needed to answer the question, using dual encoders, fused representations, and an MLP.The AMR encoder reifies labeled relations as nodes so their information is retained in the graph representations.Source: §4.2.2 · pp. 6–7
  • AMR-Multihead-Attention uses entity–relation representations as queries and AMR node embeddings as keys and values.The formulation is intended to make entity-relation interpretations responsive to sentence semantics.Source: §4.2.2 · p. 7

5. Experimental Setup

The experiments evaluate LTGen-QA and LTGen-Conv as zero-shot generation tasks across LLMs and knowledge settings. The setup specifies the models, retrieved knowledge sources, prompts, and evaluation metrics.

5. Experimental Setup

  • Both LTGen tasks are evaluated as zero-shot generation, with task instructions and external knowledge supplied through system and user prompts.Source: §5 · p. 8, §5.3 · p. 8

5.1. Large Language Models

  • The model set includes GPT-3.5 and chat-tuned LLaMA2 at 7B, 13B, and 70B, with triple ranking also tested using GPT-3.5 and LLaMA2.Source: §5.1 · p. 8

5.2. Knowledge Sources

  • Passage retrieval uses top-10 passages from the KILT Wikipedia corpus with an MSMARCO-fine-tuned Contriever; KG prompting uses top-5 relations per entity and 10 randomly selected triples per pair.Source: §5.2 · p. 8

5.3. Prompt Settings

  • The system prompt states task constraints, while the user prompt contains the question or dialogue and external non-parametric knowledge.Table 3 presents the system prompts used for both tasks.Source: §5.3 · p. 8, Table 3 · p. 8

5.4. Metrics

  • KG retrieval is scored by recall over correct entity–relation pairs relative to reference pairs.Source: §5.4 · p. 9
  • End-to-end evaluation uses KM, eKM, and rKM for both tasks, plus entailment-minus-contradiction scores for LTGen-Conv generation quality.The authors report using NLI scores because overlap metrics are not ideal for long-form text.Source: §5.4 · p. 9

6. Results and Analysis

Across LTGen, LLM performance varies with model, task, long-tail level, and required knowledge, while external knowledge generally improves results. KG triples outperform passage prompting in the reported overall comparison, and combining sources improves the LTGen-Conv NLI score without improving knowledge matching.

6.1. How LLMs Perform without External Non-parametric Knowledge?

  • GPT-3.5 performs best on LTGen-QA, while Llama 2 70B has higher knowledge-match scores on LTGen-Conv and GPT-3.5 has the higher NLI score.GPT-3.5’s E score is 0.499 versus 0.483 for Llama 2 70B, while its C score is 0.190 versus 0.267.Source: §6.1 · p. 9
  • GPT-3.5 sometimes refuses to answer rather than hallucinate, and its LTGen-Conv E-C score indicates greater robustness against hallucination at high long-tail levels.In LTGen-QA, GPT-3.5 has the lowest rKM score because it generates more unsure responses.Source: §6.1 · pp. 9–10
  • Performance generally declines as entity rarity and reference-triple count increase, although performance rises from long-tail level I to level II.Even the best GPT-3.5 model has a negative E-C score when there are more than three reference triples.Source: §6.1 · p. 10

6.2. How Different Ways of Obtaining KG Knowledge Works?

  • AMR relation linking achieves higher Recall than LLM-based approaches on both datasets with fewer than 0.2B parameters.In the oracle setting, the best AMR approach comes close to perfect Recall on LTGen-QA and exceeds 84% Recall on LTGen-Conv.Source: §6.2 · p. 10, Table 5 · p. 11
  • Relation linking performs somewhat better on LTGen-QA because its questions involve one relation, while LTGen-Conv has a larger context window.The larger context makes both entity tagging and relation linking more challenging in LTGen-Conv.Source: §6.2 · p. 11

6.3. How Different Non-parametric Knowledge Help LLMs?

  • KG triples consistently improve performance over the no-knowledge setting and outperform passage prompting in the overall comparison.Combining passages and triples does not improve knowledge-matching metrics, though it improves the LTGen-Conv NLI score.Source: §6.3 · pp. 11–12
  • Passage prompting has limited or negative effects at long-tail levels I and II, while its performance gain grows at higher levels.On LTGen-Conv, a gap separates 7B and 13B models from models of 70B and above, especially with external knowledge.Source: §6.3 · p. 12
  • With more reference triples, performance generally drops; this decline is less sharp beyond three triples when KG triples are used.The authors describe KG triples as a richer source of factual knowledge within limited context length.Source: §6.3 · pp. 12–13

7. Conclusion and Future Works

The paper introduces LTGen for evaluating long-tail grounding and finds that non-parametric knowledge improves LLM performance, particularly for rare or multi-triple knowledge.

7. Conclusion and Future Works

  • The paper introduces LTGen-QA and LTGen-Conv, datasets produced by an automatic pipeline to assess LLM grounding of long-tail knowledge.The conversational dataset involves multiple relations in a multi-turn scenario.Source: §7 · p. 13
  • Non-parametric knowledge improves LLM performance, especially for high-rarity knowledge or questions involving multiple triples.The findings span both short-form LTGen-QA and free-form conversational LTGen-Conv.Source: §7 · p. 13
  • Future work could verify and reduce grounding errors in which LLMs do not follow supplied context.Source: §7 · p. 13
  • Golden entities substantially improve KG retrieval, highlighting entity tagging as a topic for future investigation.Source: §7 · p. 14

Appendix A. Results in KILT benchmark

The appendix reports GPT-3.5's comparison with RE2G on KILT datasets and documents prompts for benchmark construction and relation linking.

Appendix A. Results in KILT benchmark

  • GPT-3.5 using only parametric knowledge outperforms RE2G with external knowledge on the TQA dataset.Table A.6 reports results for NQ and TQA development sets.Source: Appendix A · p. 14, Table A.6 · p. 14

Appendix B. Prompts

  • The LTGen-QA construction prompt asks for a natural-language question for each subject-entity and relation pair.The prompt specifies a response format containing the selected pair and generated question.Source: Appendix B · p. 14
  • The LTGen-Conv prompt requires a natural, consistent dialogue whose answers use only the listed triples as external knowledge.The user’s questions must require those triples.Source: Appendix B · p. 14
  • The relation-linking prompt asks GPT-3.5 and Llama 2 to rank candidate relations and return only the top five.Source: Appendix B · p. 15

Appendix C. Data Quality

The data-quality appendix describes annotator review of LTGen-Conv triple annotations and criteria for judging whether reference triples support a dialogue.

Appendix C. Data Quality

  • Two annotators reviewed triple annotations for 400 LTGen-Conv samples uniformly selected across long-tail levels.Source: Appendix C · p. 15
  • An annotation is correct only if its reference triples fully support the query-answer pair, require no less knowledge than the answer, and include no unused triples.Source: Appendix C · p. 15
  • The example is judged incorrect because its listed triple does not support the answer’s reasoning from the query.The query asks about Otis Clay and Ayoub Tabet, while the listed triple concerns a song’s country of origin.Source: Appendix C · p. 15

Appendix D. Experiments Details

The appendix specifies GPU resources used for LLaMA 2 prompting and AMR-based relation-linking experiments.

Appendix D. Experiments Details

Appendix D.2. Hyperparameters

Appendix D.2 documents the prompting-experiment hyperparameter setup in Table D.7 and states that unmentioned settings retain their defaults.

Appendix D.2. Hyperparameters

Loading 2405.06524v1…