PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation

Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson

arXiv:2609.08943v1cs.CLcs.AIcs.IR

TL;DR

It is unclear whether strong LLM fact-checking predictions rely on supplied evidence or parametric knowledge. This paper introduces FAE to test prediction changes under iterative evidence ablation and REAL to train evidence-dependent verification; across four benchmarks, REAL improves evidence grounding.

  • Problem

    Strong verification accuracy alone cannot show whether LLM judgments rely on retrieved evidence or memorized knowledge, leaving their evidence grounding uncertain.

    Source: §1 · p. 1

  • Method

    FAE measures evidence dependence through iterative ablation, while REAL trains verifiers with counterfactual supervision under complete and ablated evidence conditions.

    Source: §8 · p. 10

  • Results

    Across four benchmarks, REAL improves evidence-grounded verification, while current LLM verifiers often preserve predictions after supporting evidence is removed.

    Source: §1 · p. 2, §8 · p. 10

  • Takeaways & Limitations

    Strong fact-checking performance can coexist with weak evidence dependency.

    Source: §8 · p. 10

    FAE costs more than single-pass evaluation, and evidence redundancy or incomplete annotations can prevent perfect separation of evidence-grounded verification from parametric recall.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Automatic fact-checking systems assess the veracity of claims given evidence from relevant documents. Large Language Models (LLMs) have demonstrated strong performance in fact-checking due to their general reasoning capabilities. However, it remains unclear whether they faithfully make use of the evidence provided to reach veracity judgments or rely on parametric knowledge. To investigate this, we introduce Fact-Ablated Evaluation (FAE), a new evaluation framework that iteratively ablates the cited evidence to assess whether LLMs revise their predictions accordingly. Our empirical results show that current off-the-shelf LLMs as fact-checking systems rely more on their parametric knowledge than on the evidence provided. To bridge this gap between prediction accuracy and evidence grounding, we propose REAL (Rigorous Evidence Ablation Learning), a training framework that promotes evidence-dependent verification through counterfactual evidence supervision for the LLM-as-verifier models. Experiments on four fact-checking datasets across different domains demonstrate that models trained with REAL obtain superior evidence-dependent capabilities compared to standard fine-tuned models. Our findings highlight that strong fact-checking performance can still coexist with weak evidence dependency, while REAL encourages veracity predictions to remain more closely tied to the availability of supporting evidence.

1 Introduction

Although LLMs can perform fact-checking with retrieved evidence, accuracy alone cannot show whether their verdicts depend on that evidence rather than memorized knowledge. The paper introduces iterative evidence ablation to evaluate this dependence and counterfactual training to encourage it.

1 Introduction

  • Fact-checking systems are expected to provide supporting evidence alongside veracity labels, but RAG-based LLMs can draw on both retrieved evidence and memorized knowledge.Because parametric knowledge may be outdated or inaccurate, strong accuracy alone cannot establish that predictions are supported by the provided evidence.Source: §1 · p. 1
  • LLM verifiers may preserve decisions by hallucinating alternative justifications or providing no evidence, while standard metrics assess label accuracy only once.Evidence grounding therefore requires counterfactual evaluation of whether predictions change when supporting evidence is ablated.Source: §1 · pp. 1–2
  • FAE measures prediction shifts across iterative evidence ablation, revealing that current verifiers retain non-zero verdicts after supporting evidence is removed.A grounded verifier should immediately abstain once that evidence is ablated; the observed plateau indicates persistent reliance on parametric knowledge.Source: §1 · p. 2
  • REAL trains verifiers on paired inputs with and without supporting evidence, enforcing abstention after ablation to improve grounding without sacrificing fact-checking performance.The framework also uses evidence enhancement.Source: §1 · p. 2

2 Related Work

Prior work addresses LLM grounding through robustness, evaluation, and adaptation, but largely in generation-centric settings. This work instead studies fact-checking with granular labels and dynamic ablation of a model’s predicted evidence to assess its causal role in the verdict.

2 Related Work

  • Prior grounding research addresses irrelevant or insufficient context, evaluation of retrieval and generation errors, and adaptation strategies, including abstention and fine-grained evidence grounding [31] [42] [56] [72] [49] [53] [25] [28] [73].Song et al. [53] train models to abstain when evidence is insufficient, while Huang et al. [28] train models to ground claims in fine-grained textual evidence rather than document identifiers.Source: §2 · p. 2
  • Unlike generation-centric grounding, which measures semantic alignment with cited sources, this work targets fact-checking; related frameworks [24] [54] collapse REFUTE and NOT_ENOUGH_INFO into NON-SUPPORT.Source: §2 · p. 2
  • Unlike Akhtar et al. [2], whose static gold-evidence removal validates a metric, this work dynamically ablates the model’s predicted evidence to characterize its causal role in the verdict.This process can reveal when a model relies on parametric knowledge instead.Source: §2 · p. 2

3 Background

The background formalizes fact-checking as joint veracity-and-evidence prediction and describes evaluation of label correctness, label-evidence correctness, and evidence retrieval.

3.1 Problem Statement

  • A fact-checker maps a claim over sentence set S to a veracity label and supporting evidence subset, selecting them by maximizing their joint probability.Common labels are SUPPORT, REFUTE, and NOT_ENOUGH_INFO (NEI).Source: §3.1 · p. 2

3.2 Fact-Checking System Evaluation

  • Evaluation measures claim-level Label Accuracy and strict Label-Evidence Joint Accuracy, plus evidence-selection Precision, Recall, and F1 [22] [51] [57] [64] [66] [76].Strict accuracy credits a prediction only when the label is correct and at least one gold evidence sentence is identified; retrieval metrics compare predicted evidence with annotated gold evidence.Source: §3.2 · p. 3

4 FAE: Fact Ablated Evaluation

FAE iteratively removes each verifier’s predicted evidence and tracks how its judgments change as evidence availability declines. The section reports that current verifiers often retain non-NEI judgments without supporting evidence and defines metrics for measuring this behavior.

4.1 Evidence Ablation Process

  • FAE repeatedly removes the evidence predicted at each round, reruns the verifier on the remaining set, and tracks its prediction sequence for a fixed number of rounds.The sequence captures verifier behavior as cited evidence is progressively eliminated.Source: §4.1 · p. 3

4.2 Ideal vs. Actual Trajectories

  • After supporting evidence is ablated, current LLM verifiers often retain SUPPORT or REFUTE rather than switch to NOT_ENOUGH_INFO, indicating that predictions do not rely solely on provided evidence.Figure 3 contrasts ideal and observed ablation trajectories for SUPPORT/REFUTE claims on FEVER using Llama-3.1-8B-Instruct.Source: §4.2 · pp. 3–4, Figure 3 · p. 3
  • Evidence-based and knowledge-based fact-checking neuron sets overlap by 73.3% but also contain distinct subsets, supporting a neuron-level signal associated with parametric knowledge.The knowledge-based paradigm omits the evidence list; highly associated neurons are selected separately for each paradigm following prior work.Source: §4.2 · p. 4, Figure 4 · p. 4

4.3 FAE Metrics

  • FAE measures normalized accuracy across ablation rounds and defines IS for the immediate response, ER for final-round non-NEI persistence, and IO for deviation from ideal behavior across rounds.High IS indicates an immediate collapse after initial ablation; low ER indicates persistent non-NEI predictions after complete ablation; IO penalizes confidence as evidence degrades and rewards a rapid, sustained performance drop.Source: §4.3 · p. 4

5 REAL

REAL trains verifiers to condition predictions on evidence availability by pairing evidence-complete supervision with counterfactual supervision after supporting evidence is removed. It also augments evidence annotations to reduce errors caused by valid but unannotated evidence.

5.1 Evidence Ablation

  • Current LLM verifiers often retain their predictions after supporting evidence is ablated, suggesting decisions are partly sustained by parametric knowledge.Standard fine-tuning uses claim-evidence pairs under evidence-complete conditions [34] [52] [74], so persistence after ablation is not penalized when the original prediction is correct.Source: §5 · p. 4, §5.1 · pp. 4–5

5 REAL

  • REAL pairs correct verification with evidence present and abstention when supporting evidence is removed, penalizing prediction persistence under evidence ablation.This counterfactual supervision is intended to make veracity predictions depend more closely on whether the required evidence is available.Source: §5 · p. 4, §5.1 · p. 5

5.1 Evidence Ablation

  • REAL represents each instance as a claim, sentence set, and gold label, then supervises full context to produce the label and evidence, and ablated context to produce NEI with no evidence.The negative input removes the gold evidence from the sentence set; this teaches the verifier to abstain rather than preserve SUPPORT using unrelated or hallucinated evidence.Source: §5.1 · p. 5

5.2 Evidence Enhancement

  • Because benchmark evidence annotations may be sufficient but not exhaustive, ablated inputs can retain unannotated support and trigger incorrect penalties for logically valid predictions.Positive samples can also omit valid supporting evidence, weakening the reliability of counterfactual supervision.Source: §5.2 · p. 5
  • REAL augments annotations by having multiple LLMs identify supporting sentences and retaining sentences supported by a majority, reducing valid evidence left in ablated contexts.This mitigates incorrect counterfactual supervision caused by incomplete evidence annotations.Source: §5.2 · p. 5

6 Experiments

The experiments evaluate REAL against off-the-shelf and fine-tuned verifier baselines across in-domain and out-of-domain fact-checking settings. They assess veracity, evidence selection, evidence dependence, component contributions, and internal representational changes.

6.1 Models; 6.2 Hyperparameter Details

  • REAL uses Llama-3.1-8B-Instruct as its primary backbone, compares against standard supervised fine-tuning and off-the-shelf models evaluated without task-specific fine-tuning, and tests Qwen-2.5-7B-Instruct in an ablation study.Fine-tuning uses LoRA (rank r=16, scaling factor α=32, dropout 0.05 on attention projections), AdamW at 1×10−4, global batch size 32, and two epochs; outputs contain a veracity label and evidence sentence indices.Source: §6.1 · p. 5, §6.2 · p. 5

6.3 Datasets

  • Experiments cover FEVER in-domain and SciFact, Climate-FEVER, and Check-COVID out-of-domain, training REAL on FEVER only and evaluating transfer without additional training.NEI claims are excluded because they require external retrieval; SciFact's blind test split is also excluded.Source: §6.3 · pp. 5–6

6.4 Evaluation Metrics

  • Evaluation measures veracity with Label Accuracy and Strict Accuracy, evidence selection with Precision, Recall, and F1 Score, and evidence dependence with FAE metrics over four ablation rounds.The four-round setting is based on observed accuracy convergence; further ablation adds little information and often prompts hallucinated evidence identifiers.Source: §6.4 · p. 6

6.5 Research Questions

  • The experiments address veracity and grounding, domain generalisation, component and architecture ablations, and internal representational changes during evidence ablation.These questions are examined through FEVER comparisons, out-of-domain benchmarks, ablation studies, and representation-space dynamics, respectively.Source: §6.5 · p. 6

7 Results

REAL combines strong verification accuracy with predictions that respond to evidence availability, transfers evidence-grounded behaviour across domains and model families, and suppresses shifts toward parametric knowledge as evidence is removed. Its ablation results also show that evidence ablation drives grounding, while evidence enhancement improves verification and evidence selection.

7.1 Veracity and Grounding (RQ1)

  • REAL’s accuracy rapidly collapses after the first FEVER ablation round, while baselines plateau; its IS of 99.06 and IO of 99.69 exceed the best baseline’s 75.15 and 90.72.This ablation sensitivity coexists with stronger verification accuracy; REAL’s IO also exceeds Gemini-2.5-Flash-Lite’s 90.72.Source: §7.1 · p. 6, Figure 5 · p. 7
  • On FEVER, REAL raises label accuracy (83.08 to 95.66) and strict accuracy (76.27 to 94.28) over Llama-3.1-8B-Instruct, and reaches an IS score of 99.06 versus GPT-4o-mini’s 54.91.REAL also achieves the best evidence-selection recall (85.89) and F1 (78.12) among baselines.Source: §7.1 · p. 6, Table 1 · p. 7

7.2 Domain Generalisation (RQ2)

  • Across SciFact, Climate-FEVER, and Check-COVID, REAL maintains strong fact-checking performance and the strongest IS, ER, and IO scores, transferring evidence-grounded behaviour beyond FEVER.It achieves label accuracy/strict accuracy of 89.39/84.86 on SciFact and 74.86/68.69 on Climate-FEVER; out-of-domain accuracy approaches zero after about two ablation rounds, though domain mismatch and fragmented evidence prevent ideal collapse.Source: §7.2 · pp. 6–8, Table 1 · p. 7

7.3 Ablation Studies (RQ3)

  • Removing Evidence Ablation collapses FAE scores despite relatively stable initial accuracy; on FEVER, IS falls from 99.06 to 10.75 under Standard SFT and to 10.80 without Evidence Ablation.The results indicate that standard fine-tuning does not constrain predictions when evidence is missing, permitting persistence through parametric knowledge.Source: Table 2 · p. 8, §7.3 · pp. 8–9
  • Evidence Enhancement improves supervision quality rather than grounding itself: without it, Climate-FEVER evidence-selection F1 decreases from 66.68 to 41.26, while retaining Evidence Ablation preserves strong FAE scores.REAL also improves veracity prediction and FAE scores over Qwen-2.5-7B-Instruct across all datasets, indicating generalisation across model architectures and pre-training corpora.Source: Table 2 · p. 8, Table 3 · p. 8, §7.3 · p. 9

7.4 Analysis of Representation Shifts under Evidence Ablation (RQ4)

  • As evidence is ablated, the base verifier’s hidden states shift toward its parametric knowledge state, whereas REAL-trained verifiers maintain flatter alignment trajectories across benchmarks.On SciFact, base-model mean alignment rises from 0.427 to 0.456, while REAL’s remains approximately 0.430; the base and REAL confidence intervals are entirely disjoint by round two on FEVER and Check-COVID.Source: Figure 6 · p. 9, §7.4 · pp. 9–10
  • On Climate-FEVER, both models show higher proximity to the knowledge-based paradigm with overlapping confidence intervals, but REAL maintains a lower mean trajectory than the base model.The paper attributes this challenge to Climate-FEVER’s isolated sentences from multiple sources, which make robust evidentiary representations harder to form.Source: §7.4 · p. 10

7.5 Prediction Stability

  • Contradictory SUPPORT–REFUTE flips are rare, peaking at 2.98% for Llama-3.1-8B-Instruct on SciFact; REAL reduces that rate to 0.42%, supporting that FAE decay is not label instability.Instead, models generally shift toward a neutral stance as evidence is ablated.Source: Table 4 · p. 10, §7.5 · p. 10

8 Conclusion

The paper finds that strong fact-checking accuracy does not necessarily indicate evidence-grounded reasoning: LLM verifiers may preserve predictions after evidence removal, while REAL improves evidence dependence and maintains strong performance.

8 Conclusion

  • Across four fact-checking benchmarks, REAL improves evidence-grounded behavior while maintaining strong fact-checking performance in both in-domain and out-of-domain evaluation.REAL supervises verifier behavior with complete and ablated evidence through counterfactual evidence supervision and enhanced evidence annotation.Source: §8 · p. 10
  • FAE shows current LLM verifiers often preserve predictions after supporting evidence is removed, revealing substantial reliance on parametric knowledge during verification.Source: §8 · p. 10
  • Correct predictions can persist after evidence removal, so standard fact-checking accuracy alone may not indicate whether predictions are supported by retrieved evidence.Source: §8 · p. 10

Limitations

FAE is computationally expensive and best suited as a diagnostic framework, while benchmark redundancy and incomplete annotations limit how completely it can distinguish evidence-grounded verification from parametric recall.

Limitations

  • FAE’s iterative ablation and repeated verifier inference make it costlier than single-pass evaluation and currently unsuitable as a lightweight large-scale protocol.The paper presents FAE as a diagnostic framework for analysing evidence dependency.Source: Limitations · p. 10
  • Evidence redundancy and incomplete annotations can preserve correct predictions after gold-evidence removal, and REAL’s evidence enhancement cannot guarantee complete isolation of supporting evidence.These limitations make it difficult to separate evidence-grounded verification from parametric recall.Source: Limitations · p. 10

GenAI Usage Disclosure

The authors disclose using ChatGPT to improve the quality of their writing and retain responsibility for the manuscript’s content.

GenAI Usage Disclosure

  • The manuscript’s authors used ChatGPT to improve the quality of their text and remain responsible for its content.Source: GenAI Usage Disclosure · p. 11

Related papers

Loading 2609.08943v1…