Source-linked AI summary
Unified Dual-view Cognitive Model for Interpretable Claim Verification
Lianwei Wu, Yuan Rao, Yuqian Lan, Ling Sun, Zhaoyin Qi
TL;DR
Single-response evidence in interpretable claim verification can reflect biased individual cognition rather than objective reality. CICD combines collective and individual evidence and penalizes their disagreement; it outperforms strong baselines on three competitive datasets.
Problem
Claim–response evidence can inherit one-sided or biased individual cognition, limiting how objectively it reflects a claim.
Source: §1 · pp. 1–2Method
CICD generates global evidence from word- and sentence-level semantics, selects high-difference articles for local evidence, and penalizes disagreement between the two.
Source: §1 · p. 2, §3.3 · p. 5Results
CICD achieves better performance than strong baselines on three competitive datasets.
Source: §1 · p. 2Takeaways & Limitations
Experiments on three widely used datasets demonstrate CICD's effectiveness and interpretability.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Recent studies constructing direct interactions between the claim and each single user response (a comment or a relevant article) to capture evidence have shown remarkable success in interpretable claim verification. Owing to different single responses convey different cognition of individual users (i.e., audiences), the captured evidence belongs to the perspective of individual cognition. However, individuals' cognition of social things is not always able to truly reflect the objective. There may be one-sided or biased semantics in their opinions on a claim. The captured evidence correspondingly contains some unobjective and biased evidence fragments, deteriorating task performance. In this paper, we propose a Dual-view model based on the views of Collective and Individual Cognition (CICD) for interpretable claim verification. From the view of the collective cognition, we not only capture the word-level semantics based on individual users, but also focus on sentence-level semantics (i.e., the overall responses) among all users and adjust the proportion between them to generate global evidence. From the view of individual cognition, we select the top-$k$ articles with high degree of difference and interact with the claim to explore the local key evidence fragments. To weaken the bias of individual cognition-view evidence, we devise inconsistent loss to suppress the divergence between global and local evidence for strengthening the consistent shared evidence between the both. Experiments on three benchmark datasets confirm that CICD achieves state-of-the-art performance.
1 Introduction
Single-response interpretable verification can inherit biased individual cognition, leaving evidence one-sided; CICD combines collective and individual views to generate global and local evidence. Experiments on three competitive datasets show better performance than strong baselines.
1 Introduction
- The paper presents claim verification as crucial because fake news undermines trust in public institutions and democracy, while COVID-19 misinformation harms health-care systems and lives.Source: §1 · p. 1
- Earlier claim-verification methods capture content, stance, or metadata but are difficult to explain, while interpretable models interact claims with individual articles or comments.Source: §1 · p. 1
- Single-response evidence may contain one-sided or biased fragments because individual cognition does not always reflect objective reality.The paper illustrates this risk with unreliable, biased information in responses R2 and R3.Source: §1 · pp. 1–2
- CICD combines collective global evidence and individual local evidence, then penalizes disagreement to strengthen their shared evidence.CED combines word-level and sentence-level semantics to generate global evidence; ISI selects high-difference articles for local evidence.Source: §1 · p. 2
- CICD achieves better performance than strong baselines on three competitive datasets.Source: §1 · p. 2
2 Related Work
Prior interpretable verification models use claim–article interactions to extract evidence, but noisy articles and sentence-only graph representations leave limitations. CICD's hierarchical attention decoder fuses sentence- and word-level semantics for global evidence.
2 Related Work
- Interpretable methods use claim–article interactions to identify matching, consistency, implication, or conflict, but including all articles can let noisy semantics invalidate evidence.Source: §2 · p. 2
- Graph-based methods aggregate articles as sentence-level nodes, ignoring word-level semantics within each article.Source: §2 · p. 2
- CICD uses a hierarchical attention decoder to fuse sentence- and word-level semantics into fine-grained global evidence.Source: §2 · p. 3
3 The Proposed Approach
CICD uses a claim and relevant articles to form distinct inputs for its collective and individual cognition modules. CED outputs global evidence, while ISI outputs integrated local evidence fragments.
3 The Proposed Approach
- CED takes the claim and all relevant articles together, whereas ISI takes the claim with each relevant article.Source: §3 · p. 3
- CED generates a sequence of global evidence words, while ISI integrates the top-k local key evidence fragments into a vector.Source: §3 · p. 3
- The claim and article sequences use word representations obtained from a pre-trained BERT model.Article and claim representations are described as matrices of lengths l and p, respectively.Source: §3 · p. 3
3.1 Collective Cognition View-based Encoder-Decoder (CED)
CED models collective cognition by combining word-level semantics from individual responses with sentence-level semantics across responses, adjusting their contributions to generate global evidence.
3.1.1 Claim-guided Encoder
- BiL-STMs encode the claim and relevant articles, and claim attention aggregates claim information for each article word to emphasize consistent semantics and filter irrelevant noise.The filtering is intended to remove advertisements or other useless information from articles.Source: §3.1.1 · p. 3
- CED uses word-level and sentence-level representations as decoder memory, with each article’s sentence-level representation taken from its last encoded word.The word-level memory can also be represented as the words grouped by article.Source: §3.1.1 · p. 3
3.1.2 Hierarchical Attention Decoder
- Word-level attention learns salient evidence segments within each article, while sentence-level attention weights each article’s contribution to global semantics.The decoder combines these attention levels to generate a sequence rich in global evidence.Source: §3.1.2 · p. 4
3.2 Individual Cognition View-based Selected Interaction (ISI)
ISI selects articles with differing semantics and interacts them with the claim to identify local key evidence fragments from individual cognition.
3.2 Individual Cognition View-based Selected Interaction (ISI)
- ISI comprises article sentence representations, a mechanism for selecting top-k differing articles, and a co-interaction layer that explores local key evidence.The claim and each selected article interact to identify local evidence fragments.Source: §3.2 · p. 4
3.2.1 Sentence-level Representation
- BiLSTM encodes each relevant article, using its last hidden state as that article’s sentence-level representation.The encoding process is described as similar to the sequence encoding layer.Source: §3.2.1 · p. 4
3.2.2 Selected Mechanism
- The selected mechanism screens top-k articles with high semantic difference: smaller pairwise A scores correspond to more differential semantics.Larger A[m, n] entries indicate greater similarity between articles.Source: §3.2.2 · p. 4
3.2.3 Co-Interaction Layer
- The co-interaction layer lets the claim focus on each selected article and the article attend to the claim, then combines both interactions into local evidence fragments.The fragments from all top-k articles are concatenated.Source: §3.2.3 · pp. 4–5
3.3 Dual-View Classification
Dual-view classification penalizes disagreement between global and local evidence, then fuses the evidence for classification and jointly trains the losses.
3.3 Dual-View Classification
- The inconsistency loss is the KL divergence between global evidence G and local evidence I, penalizing their disagreement to strengthen consistent shared evidence.The stated purpose is to alleviate bias in individual-cognition evidence fragments.Source: §3.3 · p. 5
- The model fuses the two types of penalized evidence and applies softmax to produce a probability distribution for training with cross-entropy against the ground-truth label.Source: §3.3 · p. 5
- Joint training combines the losses as L = Loss + αLossin, with α as a hyper-parameter.The joint objective is intended to support synergy between the two cognition views.Source: §3.3 · p. 5
4 Experiments
The experiments use Snopes, PolitiFact, and FEVER, with dataset-specific labels, evaluation metrics, and data splits; model parameters are tuned on development data.
4.1 Datasets and Evaluation Metrics
- Snopes and PolitiFact contain 4,341 and 3,568 claims, respectively, while FEVER contains 185,445 claims with manually annotated Wikipedia articles.Snopes and PolitiFact associate those claims with 29,242 and 29,556 relevant articles, respectively.Source: §4.1 · p. 5
- Evaluation uses micro-/macro-F1 and class-specific precision, recall, and F1 on Snopes and PolitiFact, and accuracy and F1 on FEVER.The PolitiFact labels are grouped into true, mixed, and false; FEVER uses supported, refuted, and not enough information.Source: §4.1 · p. 5
4.2 Settings
- The study holds out 10% of Snopes and PolitiFact claims for hyperparameter tuning and uses five-fold cross-validation on the remainder; FEVER follows the train/dev/test partition of Thorne et al. (2018).Settings include 768-dimensional embeddings, 120-dimensional LSTM states, and a loss weight α of 0.2.Source: §4.1 · p. 5, §4.2 · p. 5
4.3 Experiments on Snopes and PolitiFact
CICD outperforms interpretable baselines on Snopes and PolitiFact, while ablations support contributions from its collective and individual evidence modules, inconsistency loss, co-interaction layer, and sentence-level attention.
4.3.1 Performance Comparison
- HAN consistently outperforms BERT, while BERT achieves at least 6.5% improvement on micF1 over DeClarE.The authors connect HAN’s advantage over BERT to its modeling of coherence between relevant articles.Source: §4.3.1 · p. 6
- CICD outperforms DeClarE and EHIAN, and improves on HAN and HAN-ba across most metrics, though those models gain a small precision boost on Snopes.The authors attribute CICD’s gains over DeClarE to combining word-level and holistic sentence-level features, and note that interacting with every article may introduce noise.Source: §4.3.1 · p. 6, Table 1 · p. 6
4.3.2 Ablation Study
- Removing CED or ISI degrades micF1 by 4.2%–5.5%; removing selected ISI layers reduces micF1 by at least 2.4%, and removing inconsistency loss also lowers performance.The -CED model has the largest reported drops: 5.5% and 4.6% micF1 on the two datasets; -ISI drops by 4.3% and 4.2%, respectively.Source: §4.3.2 · pp. 6–7
4.3.3 Evaluation of Co-Interaction Networks
- CoI achieves the optimal performance, exceeding other interaction methods by more than 5.1% and 5.6% in micF1, respectively, while saving 500–1,000 seconds versus Self-Att and Symm-Intr.Its time cost exceeds MLP and Biaf-Att, whose micF1 is at least 2.6% and 3.7% lower, respectively.Source: §4.3.3 · p. 7
4.3.4 Evaluation of Hierarchical Attention Decoder
- Removing any HAD component weakens performance, and sentence-level attention produces the most prominent boost.The attention-based model focuses more on high-weight sentence R4, whereas the model without sentence-level attention focuses more on R2 and R3.Source: §4.3.4 · p. 7, Table 3 · p. 7
4.4 Experiments on FEVER
On FEVER, CICD outperforms the compared pipeline and graph-based baselines in accuracy. The authors attribute the differences to evidence integration and reasoning, and to how relevant articles are weighted.
4.4 Experiments on FEVER
- CICD improves accuracy over GEAR and KGAT by at least 1.8% on development and 1.5% on testing, and outperforms NSMN and HAN by 4.3%–11.0%.The authors suggest graph-based models’ equal treatment of articles can leave biased semantics in the evidence used for reasoning.Source: §4.4 · p. 8
- CICD outperforms the pipeline baselines NSMN and HAN by 4.3%–11.0% in accuracy.The paper says these baselines lack integration and reasoning between relevant articles when capturing evidence.Source: §4.4 · pp. 7–8
- CICD improves on graph-based GEAR and KGAT by at least 1.8% on development and 1.5% on testing accuracy.The authors suggest that treating relevant articles equally can allow biased semantics to interfere with reasoning.Source: §4.4 · p. 8
4.5 Case Study: Cognition-view Explanation Analysis
The case study illustrates how CICD combines local article fragments with global evidence and constrains misleading local evidence to emphasize shared salient evidence.
4.5 Case Study: Cognition-view Explanation Analysis
- ISI excludes articles with pale, weak semantics and selects R1, R3, and R5, yielding local fragments including “this video screenshot shows,” “serious error,” and “screenshot of the video is one-sided.”The first fragment, E1, is identified as misleading and as reflecting individual-cognition deviation.Source: §4.5 · p. 8
- CED’s global sequence captures “120 million Americans a serious error” and “the screenshot is one-sided” by balancing evidence semantics across relevant articles.Source: §4.5 · p. 8
- Constraining global and local evidence disciplines misleading fragment E1 and highlights salient evidence shared between the views.Source: §4.5 · p. 8
5 Conclusion
CICD combines collective and individual cognition views to generate global and local evidence, then uses inconsistent loss to promote shared evidence. The authors report effectiveness and interpretability on three datasets and identify future extensions.
5 Conclusion
- Experiments on three different widely used datasets demonstrated CICD’s effectiveness and interpretability.Source: §5 · p. 8
- CICD uses a collective cognition encoder-decoder for global evidence and a selected interaction module for local key evidence segments.Source: §5 · p. 8
- Inconsistent loss penalizes disagreement between global and local evidence to promote consistent shared evidence.Source: §5 · p. 8
- Future work includes developing a questioning mechanism to filter suspicious evidence and integrating social cognition, psychology, and other interdisciplinary knowledge.Source: §5 · p. 8