Source-linked AI summary
Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation
Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
TL;DR
The paper asks whether strong LLM fact-checking predictions actually depend on provided evidence rather than parametric knowledge. It introduces iterative ablation and counterfactual training, finding persistent parametric reliance in current verifiers and improved evidence-dependent capabilities with REAL.
Problem
Strong fact-checking accuracy does not reveal whether LLM verifiers rely on retrieved evidence or memorised knowledge.
Method
FAE iteratively ablates model-selected evidence, while REAL trains verifiers with complete and ablated evidence through counterfactual supervision.
Results
Experiments across four benchmarks show that current verifiers often preserve predictions after support removal, while REAL improves evidence-dependent verification and maintains fact-checking performance.
Takeaways & Limitations
Strong verification accuracy can coexist with weak evidence dependency, whereas REAL ties veracity predictions more closely to supporting-evidence availability.
Takeaways & Limitations
FAE is computationally expensive, and redundant or incomplete annotations can prevent complete isolation of all supporting evidence.
Abstract
from arXiv · showhide
Automatic fact-checking systems assess the veracity of claims given evidence from relevant documents. Large Language Models (LLMs) have demonstrated strong performance in fact-checking due to their general reasoning capabilities. However, it remains unclear whether they faithfully make use of the evidence provided to reach veracity judgments or rely on parametric knowledge. To investigate this, we introduce Fact-Ablated Evaluation (FAE), a new evaluation framework that iteratively ablates the cited evidence to assess whether LLMs revise their predictions accordingly. Our empirical results show that current off-the-shelf LLMs as fact-checking systems rely more on their parametric knowledge than on the evidence provided. To bridge this gap between prediction accuracy and evidence grounding, we propose REAL (Rigorous Evidence Ablation Learning), a training framework that promotes evidence-dependent verification through counterfactual evidence supervision for the LLM-as-verifier models. Experiments on four fact-checking datasets across different domains demonstrate that models trained with REAL obtain superior evidence-dependent capabilities compared to standard fine-tuned models. Our findings highlight that strong fact-checking performance can still coexist with weak evidence dependency, while REAL encourages veracity predictions to remain more closely tied to the availability of supporting evidence.
1 Introduction
Fact-checking systems should provide both veracity judgments and supporting evidence, but strong LLM performance does not establish that predictions depend on retrieved evidence rather than memorised knowledge.
- Automated fact-checking assesses claim veracity and presents supporting evidence from external documents or knowledge graphs.The evidence helps explain system decisions and supports use in high-stakes workflows.
- Traditional fact-checking retrieves evidence before predicting veracity, while newer RAG systems provide retrieved documents as context to LLMs.
- Retrieved evidence and memorised knowledge can both influence LLM predictions, making the source of strong performance difficult to determine.Parametric knowledge may be outdated or inaccurate, so reliance on it can affect verification reliability.
- Evidence-grounded decisions should change when cited support is removed, but LLM verifiers may preserve predictions by hallucinating alternatives.The expected response to removing deterministic supporting evidence is loss of confident justification for the original prediction.
WIKIPEDIA Page
The FEVER example illustrates why evidence grounding requires counterfactual evaluation: removing supporting evidence should alter the verifier’s decision, while REAL trains abstention under that intervention.
- The example claim states that Queen is a British rock band formed in London.
- Counterfactual evaluation tests evidence grounding by observing whether a decision changes after its supporting evidence is ablated.
- FAE iteratively ablates model-selected evidence and measures decision shifts, with an evidence-grounded verifier expected to abstain after support is removed.Current verifiers instead often retain non-NEI predictions, indicating reliance beyond the provided evidence.
- REAL trains verifiers on paired inputs with and without supporting evidence, enforcing abstention when evidence is ablated.The framework is designed to create causal dependency between evidence availability and the verdict.
- The paper identifies stable predictions after removing support, introduces FAE for progressive ablation, and uses REAL to improve evidence grounding without sacrificing fact-checking performance.
2 Related Work
Prior grounding work mainly studies generation-centric settings, whereas this paper targets fact-checking with granular labels and dynamic ablation of model-selected evidence.
- Related work studies how models handle irrelevant or incomplete retrieved context, often in generation-centric tasks such as question answering.
- Generation-centric grounding commonly measures semantic alignment between free-form responses and cited sources, sometimes collapsing REFUTE and NOT_ENOUGH_INFO into NON-SUPPORT.
- This work requires granular fact-checking labels and explicit evidence dependency, using dynamic ablation rather than static gold-evidence removal.The dynamic process diagnoses whether a model actually uses the evidence for its decision.
3 Background
Fact-checking jointly predicts a veracity label and supporting evidence, while existing evaluation protocols separately assess label correctness and evidence coverage.
- A fact-checking system receives a claim and sentence set, then outputs a veracity label together with a supporting evidence subset.
- The system is formulated as maximising the joint probability of the target veracity label and evidence subset.
- The commonly used veracity labels are SUPPORT, REFUTE, and NOT_ENOUGH_INFO/NEI.
- FAE and REAL are illustrated as the paper’s proposed evaluation and training frameworks.
- Standard protocols assess veracity prediction through label metrics and evidence selection through Precision, Recall, and F1 Score.
4 FAE: Fact Ablated Evaluation
FAE repeatedly removes the verifier’s predicted evidence and tracks whether its veracity decision changes. Ideal verification should switch to NOT_ENOUGH_INFO after support is removed, but current LLM verifiers often retain non-NEI predictions, indicating incomplete evidence dependence.
- 4.1 Evidence Ablation Process: FAE iteratively removes each round’s predicted evidence and records the resulting sequence of veracity predictions.The process begins with a prediction and evidence set, then repeats ablation for a fixed number of rounds.
- 4.2 Ideal vs. Actual Trajectories: An ideal verifier should switch from SUPPORT or REFUTE to NOT_ENOUGH_INFO once its supporting evidence is ablated.Observed trajectories instead often retain non-NEI predictions across later rounds.
- 4.2 Ideal vs. Actual Trajectories: Current LLM verifiers often maintain their original decisions after supporting evidence is removed, suggesting reliance beyond the provided evidence.The paper links this behavior to parametric knowledge acquired during training.
- 4.2 Ideal vs. Actual Trajectories: FAE defines evidence-based and knowledge-based paradigms to compare verification using provided evidence with verification using parametric knowledge alone.The comparison supports analysis of whether the two modes activate overlapping or distinct neuron sets.
- 4.3 FAE Metrics: The proposed metrics IS, ER, and IO measure immediate response, final non-NEI persistence, and cumulative deviation from ideal ablation behavior.IS focuses on the first ablation step, ER on the final round, and IO on dynamics across all rounds.
5 REAL
REAL addresses the tendency of verifiers to preserve predictions after evidence removal by training on contrasting complete and ablated contexts. Its counterfactual supervision encourages correct verification with evidence and abstention when that evidence is unavailable.
- 5.1 Evidence Ablation: Standard fine-tuning uses claim–evidence pairs under evidence-complete conditions and does not explicitly train behavior after evidence removal.This can leave prediction persistence unpenalized when the original evidence is absent.
- 5.1 Evidence Ablation: REAL trains verifiers on paired full-context and evidence-ablated inputs, targeting the original label with evidence and NEI without it.The ablated context removes the gold evidence while preserving the remaining sentence set.
- 5.1 Evidence Ablation: For the Queen example, REAL removes the supporting sentence and supervises abstention rather than preservation of the original SUPPORT prediction.The remaining document context is retained after the supporting sentence is removed.
- 5.1 Evidence Ablation: REAL converts evidence availability into a counterfactual supervision signal requiring verifiers to revise predictions when support disappears.Its paired objective combines complete-evidence verification with a penalty for prediction persistence after ablation.
- 5.2 Evidence Enhancement: REAL’s ablated-input construction assumes that annotated gold evidence exhaustively captures all sentences supporting or refuting a claim.Fact-checking annotations are often designed for sufficiency, so unannotated valid evidence may remain in negative inputs.
- 5.2 Evidence Enhancement: Cross-model evidence verification retains sentences supported by a majority of GPT-4o-mini, Gemini-2.5-Flash-Lite, and Qwen-2.5-32B-Instruct.The augmentation is intended to reduce valid evidence left inside ablated contexts.
6 Experiments
The experiments compare off-the-shelf and fine-tuned verifiers, train REAL primarily on FEVER, and evaluate both in-domain and out-of-domain generalization. They use four datasets and combine standard accuracy/evidence metrics with FAE measures of evidence dependence.
- 6.1 Baselines: The study evaluates proprietary and open-source off-the-shelf LLMs without task-specific fine-tuning.Models include GPT-4o-mini, Gemini-2.5-Flash-Lite, Qwen-2.5-32B/7B-Instruct, and Llama-3.1-8B-Instruct.
- 6.1 Baselines: Standard fine-tuned baselines learn directly from claim and gold-evidence pairs, while REAL uses Llama-3.1-8B-Instruct as its primary backbone and also tests Qwen-2.5-7B-Instruct.The comparison isolates the effect of evidence-ablated training against evidence-complete supervised fine-tuning.
- 6.3 Datasets: The evaluation covers FEVER, SciFact, Climate-FEVER, and Check-COVID across in-domain and out-of-domain settings.FEVER supplies the in-domain benchmark, while the other three datasets test generalization without additional training.
- 6.2 Training and Evaluation: REAL is trained only on FEVER’s training split because its scale is intended to support generalizable evidence-grounding behavior.Out-of-domain evaluation uses SciFact, Climate-FEVER, and Check-COVID without additional training.
- 6.4 Metrics: Standard evaluation reports label and strict accuracy for veracity prediction plus precision, recall, and F1 for evidence selection.Evidence dependence is assessed with IS, ER, and IO over four ablation rounds.
- 6.5 Research Questions: The experiments ask whether REAL improves veracity and grounding, transfers evidence-dependent behavior across domains, and benefits from its individual components.These questions organize the in-domain, out-of-domain, and ablation analyses.
7 Results
REAL improves both verification accuracy and evidence dependency across in-domain and out-of-domain fact-checking, while baseline models often preserve predictions after evidence ablation. Its two components play complementary roles: Evidence Ablation drives grounding, whereas Evidence Enhancement improves supervision quality and prediction performance.
- In-domain evaluation: REAL improves FEVER label accuracy from 83.08 to 95.66 and strict accuracy from 76.27 to 94.28 versus Llama-3.1-8B-Instruct.It also achieves the strongest evidence-selection recall (85.89) and F1 (78.12) among baselines.
- In-domain evaluation: REAL achieves I_S=99.06 and I_O=99.69 on FEVER, exceeding the best baseline values of 75.15 and 90.72.Its accuracy rapidly collapses after the first ablation round, whereas baselines stabilize at non-zero plateaus.
- Out-of-domain evaluation: Across SciFact, Climate-FEVER, and Check-COVID, REAL maintains strong verification performance and the strongest FAE scores, demonstrating transfer beyond FEVER.It achieves label/strict accuracy of 89.39/84.86 on SciFact and 74.86/68.69 on Climate-FEVER.
- Baseline comparison: Baseline models trade off evidence dependency against initial accuracy, showing that stronger grounding does not naturally emerge from model scale or general reasoning capability alone.The results indicate that explicit supervision over evidence availability is required.
- Out-of-domain evaluation: REAL’s out-of-domain accuracy approaches zero after approximately two ablation rounds, although domain mismatch and fragmented evidence prevent perfectly ideal collapse trajectories.REAL nevertheless maintains substantially stronger evidence sensitivity than all baselines throughout ablation.
- Component ablations: Evidence Ablation drives grounding, while Evidence Enhancement improves prediction and evidence selection by preventing residual support from corrupting counterfactual supervision.Removing Evidence Ablation reduces FEVER I_S from 99.06 to 10.75 under Standard SFT, whereas removing Evidence Enhancement lowers Climate-FEVER evidence-selection F1 from 66.68 to 41.26.
- Representation shifts: On FEVER, SciFact, and Check-COVID, base-model representations drift toward the parametric-knowledge state as evidence is ablated, whereas REAL-trained representations remain comparatively stable.On SciFact, base-model mean alignment rises from 0.427 to 0.456, while REAL maintains approximately 0.430.
- Representation shifts: Climate-FEVER remains a limitation because its isolated sentences from multiple sources make robust evidentiary representations harder to form and increase proximity to the knowledge-based paradigm.REAL still maintains low proximity on this dataset.
8 Conclusion
The paper shows that strong fact-checking accuracy does not necessarily reflect evidence-grounded reasoning. It introduces FAE to diagnose evidence dependence and REAL to improve it while maintaining strong performance.
- Strong verification accuracy does not necessarily imply evidence-grounded reasoning in LLM-based automated fact-checking.
- FAE tests evidence dependence by repeatedly evaluating verifiers under complete and ablated evidence conditions.
- REAL trains verifiers with counterfactual evidence supervision and enhanced evidence annotation to improve evidence-grounded behaviour.
- Experiments across four fact-checking benchmarks show that REAL improves evidence-grounded behaviour while maintaining strong in-domain and out-of-domain fact-checking performance.
Limitations
FAE has higher computational costs than standard single-pass fact-checking evaluation and is therefore currently positioned as a diagnostic framework rather than a lightweight large-scale protocol. Evidence redundancy and incomplete benchmark annotations may also preserve correct predictions after evidence ablation.
- FAE requires iterative evidence ablation and repeated verifier inference, making it more computationally expensive than standard single-pass evaluation.
- FAE is currently suitable as a diagnostic framework for analysing evidence dependency rather than as a lightweight large-scale evaluation protocol.
- Evidence redundancy and incomplete annotations in existing benchmarks may allow models to preserve correct predictions after evidence ablation.
GenAI Usage Disclosure
The manuscript discloses that Generative AI was used to improve the quality of the authors’ text, while the authors remain responsible for the content.
- The manuscript used Generative AI to improve the quality of text produced by the authors.
- The disclosure identifies Chat-GPT as the Generative AI system used.
- The authors remain responsible for the manuscript’s content.