PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking with Retrieval: Faithful Large Language Model Inference

Hangfeng He, Hongming Zhang, Dan Roth

arXiv:2301.00303v1cs.CLcs.AI

TL;DR

LLM knowledge can be incomplete, outdated, or incorrect, while existing external-knowledge methods often require costly additional training or fine-tuning. RR retrieves knowledge for decomposed CoT reasoning steps as a post-processing method, and experiments show consistent gains across three reasoning tasks without additional training or fine-tuning.

  • Problem

    LLM knowledge may be incomplete, out-of-date, or incorrect, while incorporating external knowledge often requires costly additional training or fine-tuning.

  • Method

    RR uses CoT-generated decomposed reasoning steps to retrieve relevant external knowledge and support inference during post-processing.

  • Results

    RR consistently outperforms all baselines on commonsense, temporal, and tabular reasoning without additional training or fine-tuning.

  • Takeaways & Limitations

    RR is presented as a lightweight and feasible option for using external knowledge to assist LLMs.

  • Takeaways & Limitations

    GPT-3 can produce incorrect supporting facts or incorrect inferences even when its reasoning perspective is correct.

Abstract

from arXiv · show

Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting. This lightweight approach does not require additional training or fine-tuning and is not limited by the input length of LLMs. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks: commonsense reasoning, temporal reasoning, and tabular reasoning. Our results show that RR can produce more faithful explanations and improve the performance of LLMs.

1 Introduction

LLMs can perform complex reasoning through in-context learning and prompting, but their stored knowledge may be incomplete, outdated, or incorrect. RR addresses this gap by retrieving external knowledge from decomposed CoT reasoning steps without additional training or fine-tuning.

  • LLMs achieve strong performance across tasks through in-context learning without task-specific training or fine-tuning.
  • Stored LLM knowledge may be incomplete, out-of-date, or incorrect, making external knowledge important for real-world deployment.
  • RR uses CoT-generated reasoning steps to retrieve relevant external knowledge as a post-processing approach.
  • RR produces more faithful explanations and more accurate predictions without additional training or fine-tuning.
  • RR consistently outperforms all baselines across commonsense, temporal, and tabular reasoning tasks using GPT-3.

2 Related Work

Prior work improves language-model reasoning by incorporating external knowledge, adding rules or symbolic modules, and exposing latent reasoning through prompting and diverse paths. RR differs by applying external knowledge after generation to improve explanation faithfulness without additional training or fine-tuning.

  • Previous methods incorporate WordNet and ConceptNet to enhance language models for tabular reasoning.
  • Other approaches add explicit rules, integrate Wikidata with implicit LLM knowledge, or use symbolic reasoning modules to improve reasoning, coherence, or consistency.
  • RR uses external knowledge during post-processing rather than before generation, avoiding additional training or fine-tuning.
  • Chain-of-thought prompting and scratchpads expose intermediate reasoning or computation in language models.
  • Methods also sample diverse reasoning paths or train verifiers to judge reasoning outputs.

3 Rethinking with Retrieval

RR generates diverse CoT reasoning paths, retrieves knowledge for their decomposed steps, and selects the prediction most faithful to that knowledge. This design targets incorrect, missing, outdated, or misretrieved knowledge in LLM reasoning.

  • LLMs may generate incorrect supporting facts despite correctly identifying the perspective needed to answer a question.
  • Overview: RR generates diverse reasoning paths, each containing an explanation followed by a prediction, using chain-of-thought prompting.
  • Knowledge retrieval: The method retrieves relevant knowledge from a suitable knowledge base for each reasoning step or explanation sentence.
  • Faithful inference: RR estimates each reasoning path’s faithfulness from retrieved knowledge and selects the prediction most supported by that knowledge.
  • Faithful inference: In the running example, the procedure selects “So the answer is no.” because it is supported by two reasoning paths rather than one.
  • Faithful inference: The basic faithful-inference procedure has further variations described in Section 5.3.

4 Experiments

The experiments evaluate RR against prompting and self-consistency baselines across commonsense, temporal, and tabular reasoning with GPT-3 and multiple knowledge sources. RR consistently outperforms the baselines without additional training or fine-tuning.

  • Evaluation: RR is evaluated on commonsense, temporal, and tabular reasoning tasks.
  • Baselines: The baselines include standard prompting, chain-of-thought prompting, and self-consistency.
  • Commonsense reasoning: StrategyQA provides commonsense questions requiring implicit reasoning strategies, with evaluation conducted on its development set.
  • Temporal reasoning: The temporal experiment focuses on 175 single-answer implicit temporal questions from TempQuestions.
  • Temporal reasoning: For temporal reasoning, Wikidata entities and temporal relations are converted into knowledge sentences used by the inference procedure.
  • Results: RR consistently outperforms all baselines on all three reasoning tasks without additional training or fine-tuning.

5 Analysis

The analysis examines RR’s retrieval design, variants, knowledge sources, and robustness across model sizes, while identifying weaknesses in GPT-3 reasoning. Across these settings, decomposition-based retrieval and RR variants improve reasoning performance or explanation faithfulness.

  • 5.1 Limitations of LLMs in Reasoning: GPT-3 can produce reasonable explanations and predictions, but may generate incorrect supporting facts or make incorrect inferences despite identifying suitable perspectives.Examples include incorrect facts about Lil Jon’s top-ranked song and an incorrect inference about Mount Fuji.
  • 5.2 Ablation Study: Query-based retrieval performs poorly for commonsense and temporal reasoning, highlighting the importance of retrieving knowledge from decomposed reasoning steps.RR retrieves external knowledge for individual reasoning steps rather than the original query.
  • 5.2 Ablation Study: For tabular reasoning, tables alone produce significant improvement, while adding Wikidata and ConceptNet yields limited additional gains.The paper interprets this pattern as evidence that GPT-3 already captures many word-level relations in those external sources.
  • 5.3 Variations of the Proposed Approach: RR’s fact selection and fact generation variants improve explanation faithfulness and prediction accuracy relative to the basic approach and CoT prompting.The analysis compares these variants on StrategyQA using evidence paragraphs.
  • 5.3 Variations of the Proposed Approach: 79.91% accuracy is achieved by the voting-based basic approach, compared with 77.73% using retrieved paragraphs and 73.36% for self-consistency.Both basic-approach settings significantly outperform the self-consistency baseline.
  • 5.4 Impact of the Size of LMs: RR’s fact generation variant consistently outperforms CoT prompting in prediction accuracy and explanation faithfulness, even with smaller language models.The comparison uses various OPT model sizes and GPT-3 at 175B parameters.

6 Conclusion

RR is a lightweight approach for using external knowledge to assist LLMs without additional training or fine-tuning. Experiments with GPT-3 across three reasoning tasks show more faithful explanations and improved performance.

  • RR assists LLMs with external knowledge without requiring additional training or fine-tuning.The authors describe it as a lightweight and feasible option for LLMs.

A Appendix

The appendix provides additional details about the experimental setup and directs readers to the authors’ code for further information.

  • The appendix supplies additional experimental-setup details and refers readers to the code for further information.

A.1 Detailed Prompts

The experiments use task-specific prompting choices for commonsense, temporal, and tabular reasoning. The few-shot prompt uses the same exemplars as the CoT prompt without CoT reasoning.

  • Detailed Prompts: Commonsense reasoning uses the same CoT prompt as Wei et al. (2022), while temporal reasoning uses a prompt provided in Table 6.
  • Detailed Prompts: Tabular reasoning converts NLI into QA for RTE and randomly samples 6 training examples to construct the prompt.
  • Detailed Prompts: The few-shot prompt uses the same exemplars as the CoT prompt but does not involve CoT reasoning.

A.2 Description of Faithfulness Functions

Faithfulness functions score reasoning paths using supporting facts’ similarity, entailment, and contradiction signals. The selected function varies by task and evidence setting.

  • Description of Faithfulness Functions: Faithfulness functions use supporting facts’ MPNet similarity, entailment, and contradiction scores to evaluate a reasoning path.The scores are computed for all supporting facts in the explanation of a reasoning path.
  • Description of Faithfulness Functions: The experiments use function (1) for commonsense and tabular reasoning, but function (2) for temporal reasoning because contradiction scores are unreliable there.
  • Description of Faithfulness Functions: Function (3) is used for commonsense reasoning with evidence paragraphs because high-quality knowledge removes the need for complementary MPNet similarity.

A.3 Comparison of Retrieval Systems

The commonsense-reasoning experiment compared three retrieval systems for Wikipedia paragraphs. BM25 performed best in this setting.

  • 77.73%, 58.52%, and 77.29% were the performances of BM25, DPR, and BM25+DPR, respectively.
  • BM25 was the best retrieval choice for this experiment.

A.4 Implementation Details for the Two Variants of RR

The two RR variants use fact-selection and fact-generation procedures before final inference. UnifiedQA performs better with generated facts, while GPT-3 with CoT performs better with empty or gold facts.

  • Fact selection implementation details: Fact selection clusters output sentences by MPNet embedding similarity and replaces each top-ranked fact with the most faithful fact in its topic group.
  • Fact generation implementation details: Fact generation creates questions for named entities in each fact and retrieves their answers from evidence paragraphs using UnifiedQA.
  • Inference methods: UnifiedQA performs better with generated facts, whereas GPT-3 with CoT prompting performs better with empty or gold facts.
  • Inference methods: UnifiedQA is more robust to noisy inputs than GPT-3, while CoT prompting benefits the final inference step.
Loading 2301.00303v1…