PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Knowledge Guided Retrieval Augmentation for Large Language Models

Yile Wang, Peng Li, Maosong Sun, Yang Liu

arXiv:2310.05002v1cs.CL

TL;DR

LLM parametric knowledge can be incomplete, while retrieved passages may distract models from correct answers. SKR elicits self-knowledge from prior questions to adaptively invoke retrieval, and outperforms chain-of-thought and fully retrieval-based baselines across five datasets.

  • Problem

    LLM parametric knowledge can be incomplete and costly to update, while retrieved passages may negatively affect answers when they are not helpful.

    Source: §1 · p. 1

  • Method

    SKR compares answers with and without retrieval to collect self-knowledge, detects similar questions using prompting or smaller models, and adaptively retrieves external information.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    SKR outperforms chain-of-thought and fully retrieval-based baselines across five datasets, with gains of 4.08%/2.91% for InstructGPT and 4.02%/4.20% for ChatGPT.

    Source: §1 · p. 2, §4.4 · p. 6, §6 · p. 8

  • Takeaways & Limitations

    A k-nearest-neighbor strategy for eliciting self-knowledge can improve the use of both internal and external knowledge.

    Source: §3.2 · p. 4, §6 · p. 8

    The evaluation focuses mostly on general question-answering datasets and uses retrieval augmentation as the principal way to detect LLM knowledge.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) have shown superior performance without task-specific fine-tuning. Despite the success, the knowledge stored in the parameters of LLMs could still be incomplete and difficult to update due to the computational costs. As complementary, retrieval-based methods can offer non-parametric world knowledge and improve the performance on tasks such as question answering. However, we find that the retrieved knowledge does not always help and even has a negative impact on original responses occasionally. To better make use of both internal knowledge and external world knowledge, we investigate eliciting the model's ability to recognize what they know and do not know (which is also called self-knowledge) and propose Self-Knowledge guided Retrieval augmentation (SKR), a simple yet effective method which can let LLMs refer to the questions they have previously encountered and adaptively call for external resources when dealing with new questions. We evaluate SKR on multiple datasets and demonstrate that it outperforms chain-of-thought based and fully retrieval-based methods by using either InstructGPT or ChatGPT.

1 Introduction

Retrieval can complement incomplete parametric knowledge, but irrelevant or unhelpful passages may distract LLMs from correct internal answers. SKR elicits self-knowledge so models can adaptively decide when to use external retrieval, improving performance over CoT and fully retrieval-based methods.

  • LLM parametric knowledge may be incomplete and costly to update, motivating external resources as a complementary source of contextual information.Source: §1 · p. 1
  • Retrieval-augmented methods provide complementary nonparametric knowledge, but retrieved passages can distract LLMs and turn reasonable answers into incorrect ones.Figure 1 illustrates this failure mode with InstructGPT.Source: §1 · p. 1, Figure 1 · p. 1
  • Because LLMs have limited ability to recognize what they know, effective retrieval requires identifying questions where external information can improve their answers.The paper frames this ability as self-knowledge and connects it to deciding when to use a retriever.Source: §1 · pp. 1–2
  • SKR collects training-question self-knowledge by comparing answers with and without retrieval, then detects similar cases through prompting or an explicitly trained small model.The detected self-knowledge guides adaptive retrieval for new questions.Source: §1 · p. 2
  • 4.08%/2.91% and 4.02%/4.20% improvements show SKR outperforms chain-of-thought and fully retrieval-based methods across five datasets with InstructGPT and ChatGPT.The gains are reported for InstructGPT and ChatGPT, respectively.Source: §1 · p. 2

2 Related Work

Prior work improves LLM reasoning and tool use by integrating external resources, while self-knowledge research measures whether models recognize their limitations. SKR instead detects knowledge boundaries through retrieval-induced performance changes and reuses existing cases to guide retrieval.

  • Retrieval-augmented reasoning methods integrate external evidence to improve accuracy, credibility, or traceability, whereas SKR aims to balance internal and external knowledge.Examples include chain-of-thought-guided retrieval, evidence selection, retrieval feedback, and search-in-chain interaction.Source: §2 · p. 2
  • Tool-use studies teach LLMs to deploy retrievers, calculators, or other models, while SKR flexibly integrates retrievers by eliciting the model’s self-knowledge.Source: §2 · p. 2
  • Self-knowledge research evaluates confidence or recognition of unknowns, but prior work finds a gap between model and human self-knowledge.Source: §2 · p. 2
  • SKR detects knowledge boundaries through performance changes with external resources rather than introducing new evaluation metrics or challenging datasets.It also elicits self-knowledge by referring to existing cases instead of evaluating each question independently.Source: §2 · p. 2

3 Method

SKR collects examples of when an LLM can answer unaided versus when retrieval improves its answer, then uses that self-knowledge to decide whether to retrieve for new questions.

3.1 Collecting Self-Knowledge of LLMs from Training Samples

  • The method generates unaided answers with few-shot demonstrations, retrieves top-k passages from a corpus, and generates retrieval-augmented answers for comparison.The retriever finds possibly related information from corpus C, and the augmented model receives those passages with the question.Source: §3.1 · pp. 2–3
  • SKR first compares answers with and without retrieved information on training questions, forming known and unknown subsets while discarding cases where both answers are incorrect.Known questions are answered correctly without external information; unknown questions become more accurate with external information.Source: §3.1 · p. 3

3.2 Eliciting Self-Knowledge of LLMs

  • SKR elicits a new question’s self-knowledge through direct prompting, in-context demonstrations, a trained classifier, or k-nearest-neighbor search.The first two strategies use the LLM, while the latter two use smaller explicit models or a fixed encoder.Source: §3.2 · pp. 3–4
  • The k-nearest-neighbor strategy labels a target question from semantically similar training questions, assuming nearby questions induce similar LLM self-knowledge.It encodes questions, computes cosine similarity, and examines the top-k neighbors’ positive and negative labels.Source: §3.2 · p. 4

3.3 Using Self-Knowledge for Adaptive Retrieval Augmentation

  • SKR uses the elicited self-knowledge to answer known questions directly and retrieve passages only for questions identified as needing external information.This adaptive policy avoids using retrieval for every new question.Source: Figure 2 · p. 3, §3.3 · p. 4

4 Experiments

The experiments evaluate SKR across five question-answering datasets against direct, chain-of-thought, and retrieval-based baselines, using InstructGPT and ChatGPT. SKRknn achieves the strongest overall results, while other self-knowledge strategies show dataset- and model-dependent behavior.

4.1 Datasets

  • The evaluation uses five question-answering datasets spanning temporal, commonsense, tabular, strategic, and truthfulness-focused tasks.Dataset statistics, examples, and preprocessing details are provided in the appendix.Source: §4.1 · p. 5

4.2 Baselines

  • Baselines include zero-shot, few-shot, multiple chain-of-thought variants, and retrieval-based methods using retrieved passages or reasoning-step queries.The retrieval baselines include Manual-CoT-IR, IRCoT, and CoT-RR.Source: §4.2 · p. 5

4.3 Implementation Details

  • The experiments evaluate SKRprompt, SKRicl, SKRcls, and SKRknn with InstructGPT and ChatGPT, using four CoT demonstrations and top-three retrieved passages where applicable.SKRknn uses k values from 3 to 10 depending on dataset size.Source: §4.3 · p. 6

4.4 Main Results

  • SKRknn achieves the best average results across five datasets, improving over Manual-CoT and Manual-CoT-IR by 4.08%/2.91% with InstructGPT and 4.02%/4.20% with ChatGPT.The evaluation covers TemporalQA, CommonsenseQA, TabularQA, StrategyQA, and TruthfulQA.Source: §4.1 · p. 5, §4.4 · p. 6
  • SKRicl and SKRcls work inconsistently across datasets and models, while SKRprompt performs relatively poorly, indicating that direct prompting is not a reliable self-knowledge detector.SKRicl ranks second on average with InstructGPT but underperforms relevant baselines on some datasets; SKRcls is strongest on selected ChatGPT tasks.Source: §4.4 · p. 6
  • Internal reasoning and retrieved knowledge each have limitations: CoT gains are task-dependent, while retrieval helps knowledge-intensive tasks but can hurt challenging multi-hop questions.Manual-CoT does not always improve over few-shot methods, and Manual-CoT-IR improves TemporalQA and TabularQA but worsens StrategyQA.Source: §4.4 · p. 6

5 Analysis

The analysis shows how elicited self-knowledge varies with prompting strategy, training-data size, retrieval corpus, and question similarity. Across these settings, nearest-neighbor guidance and suitable external resources improve adaptive retrieval, while direct prompting is less reliable.

5.1 Effects of Different Templates for Eliciting Self-Knowledge of LLMs

  • Across all designed templates, LLMs produced positive or negative responses; for questions they thought answerable directly, accuracy was around 70%∼73%, indicating around 30% unknown unknowns.How to prompt LLMs to demonstrate reasonable confidence in their knowledge more automatically, comprehensively, and generally remains an open question.Source: §5.1 · p. 6

5.2 Effects of Elicited Self-Knowledge across Different Datasets

  • 55% to 78% beneficial guidance across datasets was achieved by SKRknn, while SKRprompt could fall below the 50% no-prior-knowledge baseline.SKRicl and SKRcls performed better than direct prompting and benefited most datasets by integrating more examples.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3 Effects of Training Data Sizes

  • Performance gradually improves as training data increases for SKRcls and SKRknn on TabularQA and CommonsenseQA.The result indicates that self-knowledge collected from existing questions is valuable and can potentially accumulate for later tasks.Source: §5.3 · p. 7, Figure 5 · p. 7

5.4 Effects of Different Knowledge Resources

  • Wikipedia gives the best average result across retrieval corpora, with a large margin over MS MARCO and SciFact on TabularQA.MS MARCO and SciFact are comparable to Wikipedia on CommonsenseQA and TruthfulQA, whose questions are more related to common knowledge.Source: §5.4 · pp. 7–8, Figure 6 · p. 8

5.5 Case Study

  • Nearest-neighbor examples identify questions where incorrect direct answers improve after retrieval augmentation, supporting adaptive use of external knowledge.In the artwork case, the model’s direct responses contained misunderstanding and hallucination, while retrieved information produced improved and correct responses.Source: §5.5 · p. 8, Table 3 · p. 8

6 Conclusion

The paper proposes SKR to elicit LLM self-knowledge and adaptively combine internal and external knowledge. Across five datasets, k-nearest-neighbor guidance performs best among the evaluated strategies and outperforms chain-of-thought and fully retrieval-based baselines.

  • SKR elicits whether LLMs know or do not know an answer, then adaptively leverages external knowledge for more accurate responses.The method includes prompting the LLMs themselves and using explicit smaller models to detect self-knowledge.Source: §6 · p. 8

Limitations

The authors identify scope and retrieval-quality limitations that constrain the current evaluation and motivate broader future work.

  • The study mainly evaluates retrieval-based knowledge detection on general question-answering datasets because of resource limitations.Future work could examine self-knowledge at memorization, understanding, and reasoning levels and evaluate broader domains; retrieval augmentation itself also needs better compatibility with LLM reasoning.Source: Limitations · p. 9

Ethics Statement

The study uses Wikipedia and five question-answering datasets to combine LLMs with external knowledge through retrieval augmentation, while focusing only on generated responses to dataset questions.

Ethics Statement

  • The evaluation uses Wikipedia as external knowledge and five publicly available question-answering datasets, focusing on generated responses rather than broader model behavior.The paper notes that retrieval augmentation may help reduce hallucination, but does not study inappropriate or misleading responses beyond the specified dataset questions.Source: Ethics Statement · p. 9

A Details of Datasets

The evaluation covers temporal, commonsense, tabular, and truthfulness-oriented question answering, using specified training and evaluation splits across five datasets.

A Details of Datasets

  • TemporalQA evaluates implicit temporal questions, while other temporal questions are used as training data.This setup follows He et al. (2022).Source: A Details of Datasets · p. 12
  • CommonsenseQA provides multiple-choice commonsense questions, whereas StrategyQA contains questions requiring multi-hop reasoning and is evaluated on its development set.The development set is used because test-set annotations are not publicly available.Source: A Details of Datasets · p. 12
  • TablularQA evaluates tabular reasoning using 1,200 development-set hypotheses extracted from Wikipedia infoboxes, while ignoring word relation triples.The evaluation uses only the questions themselves.Source: A Details of Datasets · p. 12
  • The five datasets span temporal reasoning, commonsense reasoning, tabular reasoning, and truthfulness evaluation, with dataset statistics and examples summarized in Table 4.Table 4 reports training and evaluation counts, with correct answers underlined in its examples.Source: Table 4 · p. 12, A Details of Datasets · p. 12
  • TruthfulQA measures truthfulness across domains including health, law, finance, and politics, using 113 randomly selected evaluation questions.The best answer is treated as correct, while the remaining candidates serve as options; other questions are used for training.Source: A Details of Datasets · p. 12
Loading 2310.05002v1…