Source-linked AI summary
Towards Debiasing Fact Verification Models
Tal Schuster, Darsh J Shah, Yun Jie Serene Yeo, Daniel Filizzola, Enrico Santus, Regina Barzilay
TL;DR
The paper asks whether FEVER models genuinely verify claims against evidence or exploit claim-only cues. It analyzes these biases, constructs a symmetric test set, and introduces regularization; FEVER-trained models degrade on the new set, while the proposed method supports more robust generalization.
Problem
Fact verification datasets should test claim validation against evidence, but FEVER may contain exploitable claim-only idiosyncrasies.
Method
The paper creates a symmetric test set that removes claim cues and re-weights training instances to reduce correlations between claim n-grams and labels.
Results
58.7% accuracy on the symmetric test set versus 81.8% on the original dataset shows degraded performance for the leading FEVER-trained NSMN classifier.
Takeaways & Limitations
Evaluating claim-evidence reasoning requires unbiased datasets, and regularization can support more robust generalization when training data contain bias.
Abstract
from arXiv · showhide
Fact verification requires validating a claim in the context of evidence. We show, however, that in the popular FEVER dataset this might not necessarily be the case. Claim-only classifiers perform competitively with top evidence-aware models. In this paper, we investigate the cause of this phenomenon, identifying strong cues for predicting labels solely based on the claim, without considering any evidence. We create an evaluation set that avoids those idiosyncrasies. The performance of FEVER-trained models significantly drops when evaluated on this test set. Therefore, we introduce a regularization method which alleviates the effect of bias in the training data, obtaining improvements on the newly created test set. This work is a step towards a more sound evaluation of reasoning capabilities in fact verification models.
1 Introduction
The paper shows that FEVER can reward claim-only shortcuts rather than contextual verification, then introduces symmetric evaluation data and regularization to address this bias.
- 61.7% accuracy from a claim-only BERT model exceeds the 33.3% majority baseline on FEVER.
- FEVER’s unexpectedly high claim-only performance reflects dataset-construction idiosyncrasies, including negation phrasing correlated with REFUTES independently of evidence.
- The symmetric test set manually generates pairs with preserved relations and inverse claim-evidence relations, creating combinations that prevent reliance on claim cues.
- 58.7% accuracy on the symmetric test set contrasts with 81.8% on the original dataset for the leading FEVER evidence-aware NSMN classifier.
- The paper proposes three contributions: diagnosing FEVER bias, constructing context-focused evaluation data, and introducing regularization to improve generalization under bias.
2 Motivation and Analysis
FEVER supports strong claim-only performance because dataset-construction artifacts create label-predictive cues in claims, independently of evidence. The analysis identifies negation phrasing and tests whether world knowledge explains this behavior.
- Idiosyncrasies Distorting Performance: Local Mutual Information ranks frequent claim n-grams that are highly correlated with labels, rather than favoring low-frequency n-grams.The measure is used to investigate phrase-label correlations in the training data.
- Idiosyncrasies Distorting Performance: Top REFUTES-correlated n-grams retain similar development-set correlations and mostly express strong negations introduced when annotators generated fake claims.Table 1 compares the training and development distributions for these n-grams.
- World Knowledge: 54.1% accuracy with random embeddings versus 57.3% with GloVe suggests world knowledge is not the main explanation for claim-only success.Both results remain above the 33.3% majority baseline.
3 Towards Unbiased Evaluation
The paper constructs a symmetric evaluation set to prevent models from predicting labels from claim or evidence cues alone. On this set, FEVER-trained models perform substantially worse despite complete vocabulary overlap.
- Design Goals: An unbiased verification dataset should exclude giveaway phrases and prevent systems from relying solely on world knowledge.The intended evaluation requires validating claims with respect to retrieved evidence.
- Symmetric Test Set: The symmetric test set manually generates contradictory claim-evidence pairs that preserve the original SUPPORTS or REFUTES relation.Original and generated pairs are crossed to create inverse-relations cases.
- Symmetric Test Set: By construction, any n-gram in the claim or evidence has p(l|w) = 0.5 across the test set’s two labels.This removes label-predictive cues from the inputs.
- Evaluation Scope: World knowledge may still help on the symmetric test set, but reasoning should be performed with respect to the context.Geographical locations are given as an example of potentially useful world knowledge.
4 Towards Unbiased Training
Instead of constructing a large symmetric training dataset, the paper re-weights training instances to flatten label correlations associated with giveaway claim n-grams. The procedure computes balancing weights separately from model optimization and is architecture-independent.
- Re-Weighted Training: The method increases the importance of claims with labels differing from those associated with giveaway phrases.This re-weighting aims to flatten claim n-gram and label correlations.
- Re-Weighted Training: Each training example receives an additional positive balancing weight α(i) determined by the words in its claim.The weights are assigned at the instance level.
- Bias Definition: The bias for each vocabulary n-gram wj is defined toward a class c using its association with claims and labels.The formulation is applied over the claim vocabulary.
- Optimization: The α values are optimized separately as a preprocessing step, then used to re-weight the standard training objective.This separates bias correction from model optimization.
- Implementation: The re-weighting is independent of model architecture and can be added to any objective.The paper presents this as an easily incorporated algorithmic solution to training-data bias.
5 Experiments
The experiments test whether FEVER-trained classifiers verify claims using evidence and evaluate re-weighting on a symmetric test set designed to remove claim-only cues. Models perform well on FEVER DEV but below 60% on synthetic pairs, while re-weighting improves ESIM and BERT on the new test set.
- Experimental setup: Three classifiers—NSMN, ESIM, and BERT—are evaluated on FEVER DEV and the symmetric test set, with ESIM and BERT also trained using re-weighting.BERT classifies the relation from concatenated claim and evidence; the re-weighted variants measure the regularization method’s impact.
- Experimental setup: The symmetric test set contains 956 claim-evidence pairs generated from 99 SUPPORTS and 140 REFUTES pairs.The generated pairs were sampled from cases that NSMN correctly predicted.
- Results: All models achieve less than 60% accuracy on synthetic symmetric-test pairs despite performing relatively well on FEVER DEV.The performance drop is conjectured to reflect training-data bias present in FEVER DEV but absent from the generated cases.
- Results: After re-weighting, the training distribution of give-away bigrams associated with a label becomes roughly uniform.The weights are obtained by optimizing the paper’s objective and reduce the correlation between claim phrases and labels.
- Results: The re-weighting method increases ESIM accuracy by 3.4% and BERT accuracy by 3.3% on the symmetric test set.This improvement comes at a cost in accuracy on FEVER DEV pairs, where models can exploit training-data bias.
6 Related Work
Related work shows that dataset-construction artifacts can let models perform well using incomplete inputs rather than the intended evidence or context. Prior responses include manually augmenting datasets with challenging examples.
- Dataset bias: Large-scale datasets can contain give-away phrases introduced when crowd workers use heuristics to create examples.The related work connects these artifacts to biased benchmark performance across NLP tasks.
- Dataset bias: In SNLI, entailment based solely on the hypothesis forms a very strong baseline.This parallels FEVER’s claim-only classification concern.
- Dataset bias: Reading-comprehension models can perform exceedingly well using only the question or only the referred passage on several popular datasets.This extends the incomplete-input issue beyond fact verification and natural-language inference.
- Dataset augmentation: Researchers have addressed SQuAD deficiencies by manually augmenting existing datasets with challenging examples for current systems.The cited approach provides a precedent for constructing harder evaluation cases.
7 Conclusion
The conclusion argues that FEVER contains exploitable claim-level idiosyncrasies, so evaluation must test reasoning against evidence with less reliance on those cues. It proposes symmetric evaluation alongside the existing retrieval-based pipeline and reports more robust generalization from re-weighting.
- Conclusion: FEVER contains idiosyncrasies that fact-checking classifiers can exploit to obtain high classification accuracies.The conclusion identifies these artifacts as a threat to evaluating claim-evidence reasoning.
- Conclusion: The symmetric test set makes claim-only decisions equivalent to a random guess by pairing original examples with synthetic counterparts.FEVER-trained models show degraded performance on these pairs.
- Conclusion: The re-weighting method supports more robust generalization in the presence of bias.The method is presented as the paper’s response to the bias exposed by symmetric evaluation.
- Conclusion: The authors suggest using the symmetric dataset alongside FEVER’s retrieval-based evaluation pipeline.This would assess evidence retrieval and classification as well as reasoning with respect to evidence.
B Additional Analysis
The additional analysis describes the public FEVER dataset split and points to its tabulated presentation.
- B.1 FEVER Split: The public FEVER dataset split is described in Table 6.The supplied passage identifies the table but does not provide the split values.
B.2 Top LMI-ranked Bigrams in Train and Development Set
LMI-ranked bigrams reveal label-associated cues that recur across FEVER’s training and development sets, especially for REFUTE and SUPPORT. The analysis also identifies interpretable SUPPORT and NOT ENOUGH INFO phrases that can aid development-set prediction.
- Training-set cues such as “least one” and “starred movie” also appear useful for predictions on the development set.These phrases are among the biased bigrams identified for SUPPORT and NOT ENOUGH INFO.
- Bigrams are used instead of larger patterns because they produce more comprehensible phrases for exploratory analysis.
- The analysis summarizes the top 10 LMI-ranked training bigrams for SUPPORT and NOT ENOUGH INFO.
- High Pearson correlations for the top roughly 50–100 REFUTE and SUPPORT bigrams drop around rank 400, supporting shared give-away-bigrams across splits.The NOT ENOUGH INFO curve is less stable.
B.3 Top Bigram Distribution in the Development Claims
Top LMI-ranked training bigrams occur frequently in development claims, with the strongest apparent bias in REFUTE claims. SUPPORT and NOT ENOUGH INFO claims also contain these potentially label-predictive bigrams.
- 57.6% of REFUTE development claims contain the top 1,000 LMI-ranked training bigrams, and 59.5% of those claims are labeled REFUTE.Together, these figures indicate that 34.3% of all REFUTE development claims are potentially biased.
- 34.3% of all REFUTE development claims are potentially biased by the top 1,000 LMI-ranked training bigrams.
- 32.8% of SUPPORT claims and 16.2% of NOT ENOUGH INFO claims also face the potential-bias problem.
- Figure 3 plots the percentage of claims containing at least one of the top 1,000 LMI-ranked bigrams, with colors indicating associated classes.Bar heights indicate the expected number of claims for each class: 6,666.