PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models

Haoran Wang, Kai Shu

arXiv:2310.05253v2cs.CLcs.AIcs.LG

TL;DR

FOLK addresses the need to verify claims without annotated evidence while providing justifications useful to human fact-checkers. It decomposes claims into FOL predicates and checks them against knowledge-grounded question-and-answer pairs to produce verdicts and explanations. FOLK achieves the best performance on 6 of 7 evaluation tasks.

  • Problem

    Existing zero- and few-shot claim-verification methods still require annotated evidence, while annotation is labor-intensive and models often lack justifications important to human fact-checkers.

    Source: §1 · pp. 1–2

  • Method

    FOLK decomposes claims into FOL predicates corresponding to sub-claims, checks them against knowledge-grounded question-and-answer pairs, and generates human-readable explanations.

    Source: §3.1 · p. 4, §3.3 · p. 5

  • Results

    On 6 of 7 evaluation tasks FOLK achieves the best performance, and human evaluations find its explanations consistently higher in overall quality than baselines.

    Source: §4.4 · p. 6, §5 · p. 9

  • Takeaways & Limitations

    Experiments indicate that FOL-guided claim decomposition and knowledge-grounded reasoning are effective for claim verification.

    Source: §1 · p. 2

    Experiments use synthetic claims decomposable from explicit syntax, unlike real-world claims requiring implicit semantic reasoning; costs are about $20 per 100 examples via API or 7.5 hours locally.

    Source: §6 · p. 9

Abstract

from arXiv · show

Claim verification plays a crucial role in combating misinformation. While existing works on claim verification have shown promising results, a crucial piece of the puzzle that remains unsolved is to understand how to verify claims without relying on human-annotated data, which is expensive to create at a large scale. Additionally, it is important for models to provide comprehensive explanations that can justify their decisions and assist human fact-checkers. This paper presents First-Order-Logic-Guided Knowledge-Grounded (FOLK) Reasoning that can verify complex claims and generate explanations without the need for annotated evidence using Large Language Models (LLMs). FOLK leverages the in-context learning ability of LLMs to translate the claim into a First-Order-Logic (FOL) clause consisting of predicates, each corresponding to a sub-claim that needs to be verified. Then, FOLK performs FOL-Guided reasoning over a set of knowledge-grounded question-and-answer pairs to make veracity predictions and generate explanations to justify its decision-making process. This process makes our model highly explanatory, providing clear explanations of its reasoning process in human-readable form. Our experiment results indicate that FOLK outperforms strong baselines on three datasets encompassing various claim verification challenges. Our code and data are available.

1 Introduction

The introduction frames claim verification without annotated evidence and with useful explanations as unresolved needs, then presents FOLK as a logic-guided, knowledge-grounded approach. Experiments demonstrate complex-claim verification and explanations, while the contributions emphasize annotation-free verification and symbolic decomposition.

1 Introduction

  • Existing pipelines rely on large-scale human-annotated datasets, while zero- and few-shot methods still require annotated evidence and often lack justifications.Annotation is labor-intensive and may require specialized domain knowledge; justifications matter to fact-checkers making final verdicts.Source: §1 · pp. 1–2
  • FOLK is proposed to verify claims without annotated evidence and generate natural-language explanations of its decision-making.The framework translates claims into conjunctive FOL predicates, grounds intermediate answers in external sources, and reasons over them to predict veracity.Source: §1 · p. 2
  • FOLK’s experiments demonstrate complex-claim verification with explanations and the effectiveness of FOL-guided decomposition and knowledge-grounded reasoning.The introduction reports evaluations on three fact-checking datasets spanning multi-hop, numerical, text-and-table, and open-domain scientific claims.Source: §1 · p. 2
  • The paper’s stated contributions are annotation-free claim verification and symbolic guidance for decomposition and knowledge-grounded LLM reasoning.It also presents high-quality explanations as assistance for human fact-checkers.Source: §1 · p. 2

2 Background

The background situates FOLK among claim-verification and LLM-reasoning approaches, highlighting the need for human-readable explanations and multi-step reasoning on complex claims. It distinguishes FOLK’s knowledge-grounded, open-domain verification from related methods.

Claim Verification

  • Explainable fact-checking includes attention visualizations and extractive explanations, but these either require ML knowledge to interpret or depend on costly professional-fact-checker annotations.FOLK instead explores natural-language explanations generated from reasoning steps.Source: §2 · pp. 2–3

Large Language Models for Reasoning

  • Complex claim verification often requires combining evidence through multi-step reasoning, and FOLK is framed as FOL-guided backward reasoning.Backward reasoning starts from a goal and breaks it into subgoals; the passage characterizes it as more efficient than forward reasoning.Source: §2 · p. 3
  • Prior LLM fact-verification work used much larger models and FEVER, whereas FOLK is evaluated with smaller LLMs, without training, and on scientific claims.The cited comparison describes Yao et al. (2022) as using PaLM-540B and a dataset with fewer many-hop relations and specialized-domain claims.Source: §2 · p. 3

Related Work

  • Unlike ProgramFC, FOLK uses LLMs for both decomposition and veracity prediction, supports open-domain verification, and generates explanations of its decisions.The passage contrasts ProgramFC’s closed-domain reliance on a predefined evidence corpus with FOLK’s lack of such a requirement.Source: §2 · p. 3

3 Method

FOLK decomposes claims into FOL predicates, grounds intermediate answers in external knowledge, then reasons over the grounded question-answer pairs to predict veracity and produce explanations. Predicates represent verifiable subclaims and jointly determine the claim-level verdict.

3 Method

  • FOLK’s three stages decompose a claim into FOL predicates and question-answer pairs, ground answers using external knowledge, then predict veracity and generate justifications.The overall objective is claim verification without annotated evidence.Source: §3 · p. 3, Figure 1 · p. 4

3.1 FOL-Guided Claim Decomposition

  • Each FOL predicate corresponds to a subclaim and guides a question-answer pair; the claim is supported only if all predicates are true, and refuted if any is false.A few in-context examples help LLMs identify relevant entities, relations, and facts for generating aligned questions and answers.Source: §3.1 · p. 4

3.2 Retrieve Knowledge-Grounded Answers

  • FOLK checks generated answers against external knowledge, using Google Search via SerpAPI and returning the top search result as a grounded answer.The paper notes that search results may sometimes be inaccurate, while treating Google as generally more reliable than an LLM’s internal knowledge.Source: §3.2 · p. 4

3.3 Veracity Prediction and Explanation Generation

  • For prediction, FOLK checks each predicate against its corresponding grounded answers with reasons, then predicts for the full clause and generates a human-readable explanation.Providing predicates alongside questions and grounded answers helps target which components require verification; relevant URLs are supplied for fact-checker validation.Source: §3.3 · pp. 4–5

4 Experiments

The experiments compare FOLK with claim-verification baselines across three datasets and seven challenges. FOLK leads most evaluation tasks, while ablations, smaller-model tests, and manual ratings examine its reasoning, knowledge grounding, and explanations.

4.1 Datasets

  • The evaluation spans HoVER multi-hop claims, FEVEROUS numerical, multi-hop, and text-and-table challenges, and SciFact-Open scientific claims in an open-domain setting.HoVER claims are grouped by hop count; SciFact-Open uses complete-evidence claims labeled SUPPORT or REFUTE.Source: §4.1 · p. 5

4.4 Main Results

  • FOLK achieves the best performance on 6 of 7 evaluation tasks; on HoVER, it beats baselines by 7.37% on three-hop and 7.94% on four-hop claims.Its performance increase from three-hop to four-hop claims is 10.13%, compared with ProgramFC’s 3.68%.Source: Table 2 · p. 6, §4.4 · p. 6

4.2 Baselines / 4.3 Experiment Settings

  • Compared baselines were Direct, Chain-of-Thought, Self-Ask, and open-book ProgramFC; FOLK and baselines used GPT-3.5 text-davinci-003 (175B), with 4–6 prompts across datasets sampled from random train and development examples.ProgramFC was excluded from SciFact-Open because it cannot perform open-domain claim verification.Source: §4.2 · p. 5, §4.3 · p. 6

4.4 Main Results

  • On FEVEROUS, FOLK outperforms baselines by 7.52%, 9.57%, and 2.69% on its three tasks, including numerical reasoning and reasoning over text and tables.The paper presents these results as evidence that FOLK performs beyond multi-hop reasoning tasks.Source: §4.4 · p. 6
  • FOLK beats CoT and Self-Ask by 11.30% on average and outperforms Direct, which relies on uncontrolled LLM knowledge.The paper notes that Direct may produce a correct prediction for incorrect reasons when some claim predicates are false.Source: §4.4 · p. 6

4.5 The Impacts of FOL-Guided Reasoning

  • With identical knowledge-grounded answers, FOLK consistently outperforms CoT and Self-Ask across all three HoVER tasks.The ablation retains each baseline’s prompt format while using FOLK-generated questions and answers, isolating the contribution of FOL-guided reasoning.Source: §4.5 · p. 7

4.6 The Impacts of Knowledge-Grounding

  • FOLK performs better with knowledge-grounded answers, and the paper reports that more accurate search can lead to better performance.The grounding ablation contrasts LLM-generated answers with retrieved answers; a separate study restricts search to Wikipedia.Source: §4.6 · p. 7

4.7 The Generalization on Different-sized LLMs / 4.8 Assessing the Quality of Explanations

  • FOLK outperforms CoT and Self-Ask across model sizes except on three-hop claims with llama-13B, and its explanations rank best for coverage, soundness, and readability.FOLK with llama-30B is comparable to ProgramFC using GPT-3.5 (5.8× larger) on three- and four-hop claims. In manual evaluation, FOLK’s MAR improves by 0.16 on coverage and 0.40 on readability; lower MAR is better.Source: Figure 2 · p. 7, §4.7 · p. 8, §4.8 · p. 8

5 Conclusion

FOLK uses First-Order Logic to decompose complex claims for knowledge-grounded LLM reasoning, addressing the scarcity of annotated datasets and explanations. Experiments report promising performance on three challenging datasets and stronger explanation quality than baselines.

5 Conclusion

  • FOLK uses First-Order Logic to guide LLMs in decomposing complex claims into sub-claims verified through knowledge-grounded reasoning.Source: §5 · p. 9
  • FOLK shows promising performance on three challenging datasets with 4–6 in-context prompts and no additional training.With a medium-sized LLM such as llama-30B, it makes accurate predictions and generates explanations.Source: §5 · p. 9
  • Three human annotators’ evaluations found that FOLK consistently outperforms baselines in explanation overall quality.Source: §5 · p. 9

6 Limitations

The experiments use synthetic claims that can be decomposed through explicit reasoning based on syntactic structure, whereas real-world claims may require implicit reasoning. FOLK also has substantially higher computational costs than supervised claim verification methods.

6 Limitations

  • Experimental claims are synthetic and explicitly decomposable from their syntactic structure, limiting the evaluated claim setting.Source: §6 · p. 9
  • Real-world claims often have complex semantic structures requiring implicit reasoning, leaving this gap an important direction for future work.Source: §6 · p. 9
  • FOLK costs around $20 per 100 examples through the OpenAI API or about 7.5 hours on locally deployed llama-30B models using an 8x A5000 cluster.Source: §6 · p. 9

7 Ethical Statement

The ethical statement discusses possible biases, misuse potential, and environmental impact associated with the models and their computational demands.

7 Ethical Statement

  • The authors acknowledge possible biases in language-model training data and in some factuality assessments, which are beyond their control.Source: §7 · p. 9
  • The models may be susceptible to misuse by malicious individuals, so the authors urge cautious use.Source: §7 · p. 9
  • Large language models demand substantial computation and GPU/TPU use, contributing to global warming, although this approach uses few-shot in-context learning rather than training models from scratch.Source: §7 · p. 9

A Implementation Details about the Baselines

This section gives implementation details for the three baselines used in the work.

A Implementation Details about the Baselines

A.1 Large Language Models

The appendix identifies the LLaMA models and weight formats used in the experiments, noting a precision difference driven by limited GPU memory.

A.1 Large Language Models

  • The experiments use LLaMA models from Decapoda Research, whose converted weights are not the official model weights.The weights were converted to work with Transformers.Source: §A.1 · p. 12
  • LLaMA-13B is loaded at its original float16 precision.Source: §A.1 · p. 12
  • LLaMA-30B is loaded in int8 precision because of limited GPU memory.Source: §A.1 · p. 12

A.2 ProgramFC

The authors implement ProgramFC from the authors’ provided code and adapt it to accept their custom data input.

A.2 ProgramFC

  • ProgramFC is implemented using code provided by its authors.Source: §A.2 · p. 12
  • The implementation is modified to accept the paper’s custom data input.Source: §A.2 · p. 12

B Prompts

This appendix section presents prompt templates and examples for claim decomposition and reasoning toward supported or not-supported judgments, contrasting ordinary question-based prompts with FOLK’s predicate-guided process of querying supporting facts and evaluating predicates and their conjunctions to produce verdicts with explanations.

Listing 1: Chain-of-Thought (CoT) Decompose Prompt

  • The CoT decomposition prompt asks for the necessary questions to verify a claim and illustrates this with claim-specific follow-up questions.Examples break claims into questions about locations, events, and people’s attributes.Source: B Prompts · p. 13

Listing 2: FOLK Decompose Prompt

  • FOLK decomposition prompts ask the model to define claim predicates, turn them into follow-up questions, and answer those questions.Examples represent claims with predicates such as Location, Population, and Commit, then provide corresponding questions.Source: B Prompts · pp. 13–14

Listing 3: Direct Reasoning Prompt

  • The direct reasoning prompt asks the model to verify a claim and explain its decision, with examples returning supported or not-supported labels and reasons.The examples include both outcomes and explanations based on claim-relevant facts.Source: B Prompts · p. 14

Listing 4: CoT Reasoning Prompt

  • The CoT reasoning prompt asks whether a claim is supported or not and directs the model to think step by step before giving its answer and reasons.Its examples show intermediate factual reasoning followed by a label and explanation.Source: B Prompts · pp. 14–15

Listing 5: Self-Ask Reasoning Prompt

  • The Self-Ask prompt uses a question and context to request a supported or not-supported judgment with an explanation.Its example context is organized as follow-up questions and answers before the final judgment.Source: B Prompts · p. 15

Listing 6: FOLK Reasoning Prompt

  • The FOLK reasoning prompt supplies predicate-based context and asks for a supported or not-supported answer with an explanation.Its example evaluates predicates individually and combines them to determine the claim’s prediction.Source: B Prompts · p. 16

B Prompts

  • The New York example marks a claim [NOT_SUPPORTED] while its explanation distinguishes the Rangers’ popularity from Madison Square Garden’s approximately 19,500 capacity.The claim context gives the venue capacity as 70,000, which the explanation contrasts with approximately 19,500.Source: B Prompts · p. 16
  • The Werner Gunter Jaffé Fellner example uses questions about his birthplace and Frankfurt’s state to support both predicates and their conjunction.The prompt concludes that the claim is [SUPPORTED] and explains the result in natural language.Source: B Prompts · p. 16
  • The Tom Jones example rejects a compound claim when its birth-year and nationality predicates are false, despite a true co-authorship predicate.The prompt concludes [NOT_SUPPORTED] and explains the birth and nationality mismatches alongside the true co-authorship fact.Source: B Prompts · pp. 16–17

C Manual Evaluation Example

The manual evaluation asks raters to rank three explanations by coverage, soundness, and readability. An example contrasts explanations of the same claim, with differing levels of detail.

C Manual Evaluation Example

  • Explanations are ranked on coverage, soundness, and readability, with ties allowed.Coverage concerns including salient information; soundness requires logical consistency with the claim; readability concerns clarity and coherence.Source: C Manual Evaluation Example · p. 17
  • The example evaluates three explanations for the claim about Anat Berko’s election to the Knesset and Likud winning 30 seats.Source: C Manual Evaluation Example · p. 17
  • The explanations vary in detail: one mentions the November 2015 elections, another identifies Berko as a Likud candidate, and another adds her list position and a rival party’s seat count.Source: C Manual Evaluation Example · p. 17

Related papers

Loading 2310.05253v2…