PaperlayerView exact revision on arXiv

Source-linked AI summary

DialFact: A Benchmark for Fact-Checking in Dialogue

Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, Caiming Xiong

arXiv:2110.08222v2cs.CL

TL;DR

Dialogue fact-checking is relatively unexplored, and conversational claims introduce colloquialisms, ellipsis, coreference, and ambiguity that complicate verification. The paper introduces fact-checking in dialogue and proposes DIALFACT as an evaluation dataset. Existing fact-checking models trained on non-dialogue data fail to perform well on DIALFACT, while automatically generated responses provide weak supervision that improves performance.

  • Problem

    Dialogue fact-checking is relatively unexplored, and conversational claims introduce colloquialisms, ellipsis, coreference, and ambiguity that complicate verification.

  • Method

    The paper introduces fact-checking in dialogue and proposes DIALFACT as an evaluation dataset.

  • Results

    Existing fact-checking models trained on non-dialogue data fail to perform well on DIALFACT, while automatically generated responses provide weak supervision that improves performance.

  • Takeaways & Limitations

    DIALFACT provides a benchmark for advancing fact-checking, consistency modeling, and evaluation in dialogue.

  • Takeaways & Limitations

    DIALFACT covers only a specific domain and uses Wikipedia as its background evidence source.

Abstract

from arXiv · show

Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation. We introduce the task of fact-checking in dialogue, which is a relatively unexplored area. We construct DialFact, a testing benchmark dataset of 22,245 annotated conversational claims, paired with pieces of evidence from Wikipedia. There are three sub-tasks in DialFact: 1) Verifiable claim detection task distinguishes whether a response carries verifiable factual information; 2) Evidence retrieval task retrieves the most relevant Wikipedia snippets as evidence; 3) Claim verification task predicts a dialogue response to be supported, refuted, or not enough information. We found that existing fact-checking models trained on non-dialogue data like FEVER fail to perform well on our task, and thus, we propose a simple yet data-efficient solution to effectively improve fact-checking performance in dialogue. We point out unique challenges in DialFact such as handling the colloquialisms, coreferences and retrieval ambiguities in the error analysis to shed light on future research in this direction.

1 Introduction

The paper introduces dialogue fact-checking and the DIALFACT benchmark, addressing dialogue-specific difficulties such as informal claims, colloquialisms, ellipsis, and coreference. DIALFACT supports verifiable-claim detection, Wikipedia evidence retrieval, and claim verification, with annotated claims and weakly supervised training data.

  • Motivation: Fact-verification tools are motivated by the risk that humans and generative systems can spread false or outdated information at scale.The paper highlights misinformation risks during public-health crises such as COVID-19.
  • Motivation: Dialogue claims are difficult to verify because they are often informal, sparse in factual content, and mixed with opinions, slang, and colloquialisms.Unlike formal claims in existing datasets, dialogue utterances may require distinguishing factual content from conversational language.
  • Motivation: Ellipsis and coreference make dialogue utterances incomplete or ambiguous, requiring fact-checking systems to use dialogue context and explicit reasoning.Humans can often resolve absent information from context, whereas systems may need to model this behavior directly.
  • Contribution: DIALFACT introduces dialogue fact-checking through verifiable-claim detection, Wikipedia evidence retrieval, and supported/refuted/NEI claim verification.The benchmark uses both human-written and machine-generated claims from Wizard of Wikipedia dialogues.
  • Contribution: Each response claim and its Wikipedia evidence sentences receive crowd annotations with rigorous quality checks, while weakly supervised data is created using claim-generation techniques.The proposed techniques include negation, entity swapping, language-model mask-and-fill, and knowledge-grounded generation.

2 Related Work

Prior work studies fact-checking across several media and dialogue consistency, but existing resources do not fully address conversational claims with dialogue context. DIALFACT targets retrieval and verification challenges arising from ambiguity and coreference in dialogue.

  • Fact-checking and dialogue consistency: Fact-checking research has used Wikipedia passages, tables, social media, and news articles, while dialogue work has focused on factual consistency or consistent response generation.These lines of work address related problems but use different media or objectives.
  • Dialogue fact-checking: Colloquial Claims converts FEVER claims into colloquial style but lacks dialogue contexts and includes only verifiable claims.DIALFACT additionally requires resolving ambiguities and coreferences during evidence retrieval and claim verification.
  • Synthetic datasets: Synthetic-data research has generated adversarial or difficult claims to improve evaluation robustness and test-set complexity.The cited work includes synthetic datasets and automatically generated claims for fact-checking.

3 Task Background

Dialogue fact-checking evaluates the final utterance of a conversation through a pipeline that identifies verifiable claims, retrieves evidence, and assigns supported, refuted, or NEI labels. DIALFACT builds these categories from Wizard of Wikipedia conversations, including claims that require constructing REFUTED and NEI examples.

  • Task definition: The task fact-checks the final utterance u_n of a conversation, treating it as claim c within its dialogue context.The conversation context is represented as a list of utterances C = {u_1, u_2, ..., u_n}.
  • Task definition: The pipeline first classifies a claim as VERIFIABLE or NON-VERIFIABLE based on whether it contains factual information checkable against Wikipedia.Personal opinions and personal information are examples of NON-VERIFIABLE content.
  • Task pipeline: The system then retrieves documents and evidence sentences before assigning SUPPORTED, REFUTED, or NOT ENOUGH INFORMATION according to the evidence.NEI applies when the available evidence cannot validate or refute the claim.
  • Label construction: VERIFIABLE claims may receive any of the three verification labels, whereas NON-VERIFIABLE claims are always assigned NEI.DIALFACT uses Wizard of Wikipedia, where wizards select knowledge sentences or indicate that no sentence was used; additional construction is needed for REFUTED and NEI claims.

4 Dataset Construction and Annotation

DIALFACT combines automatically generated and human-written dialogue claims with crowd-annotated Wikipedia evidence, using multiple transformations and quality-control rounds to construct a challenging benchmark.

  • Claim generation: DIALFACT creates both automatically generated and human-written claims to represent claims produced by dialogue systems and humans.Automatic generation includes negation, entity and adjective substitution, mask-and-fill, and dialogue-model generation.
  • Evidence construction: Claims are paired with candidate Wikipedia evidence retrieved through entity and noun-phrase extraction, MediaWiki search, and SpaCy and BM25 ranking.The final evidence set also includes the original Wizard of Wikipedia knowledge sentence when it is not already present.
  • Claim and evidence annotation: Crowd workers label verifiability, add evidence when needed, and assign SUPPORTED, REFUTED, or NEI labels across three annotation rounds.The final label uses majority voting, while evidence is the union of evidence selected across rounds; 5% of incoherent automatically created claims were removed.
  • Dataset statistics: The test set contains 3,760 dialogue contexts averaging 3.1 claims each, while the validation set contains 3,738 contexts averaging 2.8 claims each.Average claim length is 22.0 tokens in test and 20.0 in validation, with 1.3 and 1.1 evidence sentences per claim, respectively.
  • Quality control: Quality control used qualified Mechanical Turk workers, feedback and automatic checks, with extra annotation rounds collected for agreement assessment.Workers had at least 5,000 completed HITS and a 95% or higher acceptance rate; 87 workers participated in the main collection stage.
  • Quality control: Top-bigram analysis finds no obvious lexical bias from negations, and substantial overlap across categories suggests labels lack simple lexical shortcuts.Written REFUTED claims show somewhat higher association with negation-related bigrams than generated claims, although their LMI values were not high.

5 Experiments

The experiments establish baselines for verifiable claim detection and compare simple lexical and dialogue-inference signals for distinguishing verifiable from non-verifiable responses.

  • Experimental setup: The experiments cover verifiable claim detection, evidence retrieval, and claim verification as the three dialogue fact-checking subtasks.
  • Verifiable claim detection: Verifiable claim detection compares lexical overlap, DNLI neutral-class probability, their combination, and a random classifier.Thresholds for VERIFIABLE versus NON-VERIFIABLE predictions are selected using validation data, with accuracy and per-class F1 reported.

5.2 Evidence Retrieval

Evidence retrieval separates document retrieval from sentence selection and evaluates WikiAPI and DPR variants with or without dialogue context.

  • Document retrieval: WikiAPI methods outperform DPR-based methods on document recall, and Table 4 reports the corresponding test-set comparison.A document counts as relevant when it contains a gold evidence sentence.
  • Document retrieval: Adding dialogue context improves document retrieval for both WikiAPI and DPR, helping resolve dialogue-dependent retrieval needs.WikiAPI leverages context through queries combining the last two dialogue turns with the response claim, while DPR has context-aware variants.
  • Document retrieval: DPR often retrieves documents about the correct topic but fails to retrieve a relevant evidence sentence, highlighting the importance of entity linking for dialogue fact-checking.
  • Evidence sentence selection: Evidence sentence selection ranks the union of sentences from retrieved documents with a fine-tuned BERT model trained to distinguish relevant from irrelevant evidence.The selector uses Wizard of Wikipedia context-response pairs and hard negatives in addition to randomly selected evidence sentences.

5.3 Claim Verification

Claim verification classifies dialogue claims as SUPPORTED, REFUTED, or NEI using dialogue context and retrieved evidence. Aug-WoW outperforms baselines, but retrieval quality remains critical and substantial reasoning challenges persist.

  • Results: Aug-WoW outperforms all baselines across oracle, Wiki-Evidence, and DPR-Evidence settings.Table 6 reports Accuracy and Macro F1 for the three evidence settings.
  • Results: Retrieved evidence lowers every baseline’s performance relative to oracle evidence, while even oracle-supported models remain below 70% accuracy.The result highlights evidence retrieval as an important bottleneck for claim verification.
  • Results: Aug-WoW performs strongly on both generated and written claims, although generated claims are more difficult for all models.Generated claims were selected partly according to existing-model difficulty, making them more challenging than written claims.
  • Ablations: Aug-WoW-noctx is comparable to Aug-WoW, whereas the larger Bert model is more sensitive to evidence quality and performs poorly with retrieved evidence.The ablation results show that increasing the base model size does not ensure robustness to imperfect retrieval.
  • Ablations: A claim-only Aug-WoW model reaches 33.2% accuracy and 28.9% macro F1, indicating that lexical cues alone do not yield strong DIALFACT performance.The experiment removes evidence during both training and testing.
  • Discussion: Models are frequently misled by lexical overlap and struggle with complex, commonsense, and colloquial reasoning during verification.Examples include incorrect SUPPORTED predictions for overlapping claims and failures on evidence requiring semantic reasoning.

6 Conclusion

The paper introduces DIALFACT, a benchmark for dialogue fact-checking with three tasks and Wikipedia evidence. It also provides annotations, weakly supervised data, baselines, and error analysis showing that the task remains challenging.

  • 6 Conclusion: DIALFACT is a dialogue fact-checking benchmark with verifiable claim detection, evidence retrieval, and claim verification sub-tasks.Claim verification predicts whether a response is supported, refuted, or not sufficiently supported by the dialogue and retrieved evidence.
  • 6 Conclusion: The benchmark uses qualified crowd workers to annotate response categories and corresponding evidence, while weakly supervised data are created with automated claim-generation techniques.The generation methods include contradiction, infilling, substitutions, negation, entity swapping, mask-and-fill, and knowledge-grounded generation.
  • 6 Conclusion: Existing fact-checking models trained on non-dialogue data perform poorly on DIALFACT, while automatically generated responses provide weak signals that improve performance.The authors establish baselines and identify remaining weaknesses through empirical analysis.

Ethical Considerations & Broader Impact

DIALFACT aims to support more trustworthy dialogue systems and monitoring against misinformation, but its evidence and coverage impose important boundaries. The benchmark is domain-specific, may contain residual labeling or bias issues, and should not be treated as universal.

  • Ethical Considerations & Broader Impact: DIALFACT could support more accurate fact-checking systems and metrics, contributing to dialogue systems that are more faithful to factual knowledge.The paper also identifies monitoring conversations and providing accurate information as potential real-world uses.
  • Ethical Considerations & Broader Impact: The benchmark covers a specific domain with Wikipedia as background knowledge, limiting its direct scope across domains and evidence sources.The authors caution against treating it as a universal tool for all domains and scenarios.
  • Ethical Considerations & Broader Impact: Residual incorrect labels and biases may remain despite quality controls, creating risks when models are deployed beyond the benchmark’s covered domains or with unreliable resources.The authors mitigate some risk by using Wikipedia and curating difficult training and testing instances, while noting that broader validation remains necessary.

A Supplementary Results

Supplementary experiments examine validation and test-set claim verification, alternative label handling, and Aug-WoW’s confusion patterns. Performance trends remain similar across settings, with Aug-WoW strongest in the two-way evaluation and weakest on NEI claims.

  • A Supplementary Results: Aug-WoW performs best when REFUTED and NEI are merged into NOT-SUPPORTED for two-way test-set classification.All baselines improve because the task changes from three-way to two-way classification.
  • A Supplementary Results: Human annotators sometimes confuse REFUTED and NEI or miss Wikipedia evidence that refutes a claim, motivating the merged-label experiment.Similar difficulties were reported in other fact-checking tasks.
  • A Supplementary Results: Removing NEI-PERSONAL non-verifiable claims while retaining only verifiable NEI claims produces trends similar to the main test-set results.The corresponding results are reported in Table 11.
  • A Supplementary Results: Aug-WoW has its lowest performance on NEI claims and confuses NEI with REFUTED most often.The confusion matrix in Figure 2 highlights this specific error pattern.

B Implementation Details

The implementation uses established generation, retrieval, and verification components, with practical adaptations for dialogue and Wikipedia-version differences. Reported experiments include validation metrics and retrieval settings, while exact numerical table values are not supplied here.

  • B Implementation Details: Claim-generation implementations include FEVER-2 negation rules, Hugging Face T5 mask-and-fill, and BlenderBot generation fine-tuned on the full Wizard-of-Wikipedia training set.The reported training setup includes batch size 40 for BlenderBot and validation-set model selection.
  • B Implementation Details: WikiAPI retrieval filters colloquial filler words such as “I,” “Yes,” and “They” from candidate entities, improving the quality of returned documents.The filtering uses a manually created stopword list before calling the MediaWiki API.
  • B Implementation Details: DPR retrieval uses 21M Wikipedia passages split into disjoint 100-word blocks and retrieves the top 100 documents per claim.The base encoder is fine-tuned using constructed data for claim-only and context-aware variants.
  • B Implementation Details: Evidence matching is relaxed from exact document matching to partial matching because the Wikipedia versions used for gold evidence and retrieval differ.Wording changes and edits can prevent exact matches even when the retrieved document is substantively correct.
  • B Implementation Details: Claim verification uses VitaminC, CorefBERT, Aug-WoW, and Colloquial model implementations drawn from released repositories and codebases.The supplementary tables report Accuracy and Macro F1 for claim verification, but their cell values are not included in the supplied passages.

C AMT Instructions

The annotation protocol asks workers to distinguish verifiable from non-verifiable responses and label factual responses as supported, refuted, or not enough information using selected Wikipedia evidence. Instructions also separate personal or generic responses from encyclopedic claims and require careful, evidence-based labeling.

  • Refuted: Refuted responses contain factual information that contradicts the evidence, and even a small incorrect part is sufficient for the refuted label.The Jazz example is refuted because it describes roots as American rather than African-American.
  • Not Enough Information: Not Enough Information applies when the provided Wikipedia evidence cannot verify or refute the response, while general knowledge may still be assumed.Examples include claims about worldwide jazz distribution or unclear waterfall names that the supplied evidence does not establish.
  • Worker instructions: Workers must read context carefully, write at least nine words when creating or editing responses, base labels exclusively on the response and selected evidence, and avoid spelling errors or conversational fillers.The protocol specifically disallows “know-it-all” phrases such as “did you know.”
  • Personal/generic response: Personal or generic responses may include speaker-specific opinions or experiences but must not make factual claims verifiable through encyclopedic knowledge.The examples treat hunting preferences and subjective views about foster-care arrangements as personal content even when related evidence exists.
  • Claim labeling: Workers label responses as SUPPORTED, REFUTED, or NOT ENOUGH INFORMATION after reading the dialogue, response, and related Wikipedia evidence.Supported and refuted labels require at least one selected or copied Wikipedia sentence as evidence.
  • Verifiability: A response is verifiable when Wikipedia evidence could be retrieved to reduce uncertainty about whether its statement is true or false.Responses containing only personal opinions or personal information without verifiable factual content are marked separately.
Loading 2110.08222v2…