Bootstrapping Multi-view Representations for Fake News Detection

Introduction

• Because many social-network posts are multimodal, unimodal features alone are described as insufficient, while manual removal of false news is laborious and expensive.The introduction motivates automatic detection as a way to analyze information and help administrators block its spread.Source: Introduction · p. 1
• The paper identifies unresolved questions about how views affect decisions and whether cross-modal correlation reliably signals news fidelity.It notes that both real and fake news can contain cross-modal correlations and that many existing models do not expose the most contributive components.Source: Introduction · p. 1
• BMR extracts text, image-pattern, and image-semantics views, uses single-view predictions to reweigh them, and bootstraps the representations for detection.Its examples report single-view predictions, cross-modal consistency predictions, and the final detection result.Source: Introduction · p. 1, Figure 1 · p. 2
• BMR outperforms state-of-the-art multimodal fake-news detection schemes on popular datasets and provides a mechanism to interpret representation contributions.The paper frames this as one of three contributions alongside multi-view representations and disentangling unimodal and multimodal information.Source: Introduction · p. 2

Related Works

• Unimodal text- and image-based approaches do not detect cross-modal correlations, limiting their fit to multimodal news.The related-work discussion names text, image-distribution, image-manipulation, and spatial/frequency approaches.Source: Related Works · p. 2
• Existing multimodal systems fuse features through integration, co-attention, or concatenation, leaving the contribution of each view difficult to measure.BMR is contrasted as reweighing views based on single-view classifications and adaptively bootstrapping them.Source: Related Works · p. 2
• Prior consistency-learning methods use consistency scores or staged contrastive learning, while the paper questions fixed feature weighting and possible loss of learned cross-modal knowledge.CAFE adjusts feature weights using consistency; the authors note it may penalize unimodal features when consistency is high, while CMC's fine-tuning may erase cross-modal knowledge.Source: Related Works · p. 2

Proposed Approach

• BMR uses four branches—image pattern, image semantics, text, and fusion—across feature extraction, refinement and fusion, disentangling and reweighing, and bootstrapping.The final decision uses bootstrapped multi-view representations.Source: Proposed Approach · p. 2, Figure 2 · p. 3

Multi-view Feature Extraction

• The extraction stage produces coarse image-pattern, image-semantic, and text representations using BayarConv and InceptionNet-V3, MAE, and BERT, respectively.BayarConv suppresses image components contributing largely to semantics before InceptionNet-V3 processing.Source: Multi-view Feature Extraction · p. 3

Feature Refinement and Fusion with iMMoE

• iMMoE refines image-semantic and text features and generates a fused feature; token attention aggregates representations, while gates weight expert outputs.The revised gates are not constrained to positive weights summing to one.Source: Feature Refinement and Fusion with iMMoE · pp. 3–4

Disentangling & Reweighing

• Single-view fidelity scores reweight the three unimodal views, while consistency learning trains on real text-image pairs as positives and shuffled pairs as negatives, and reweights a separate irrelevance token.The consistency task uses regression supervision and can be learned in parallel with the main task.Source: Disentangling & Reweighing · p. 4

Bootstrapping Stage and Loss Function

• A second iMMoE bootstraps five reweighed views for final classification, trained with final BCE plus averaged single-view and consistency losses using α=1 and β=4.The single-view loss averages the text, image-pattern, and image-semantic prediction losses.Source: Bootstrapping Stage and Loss Function · p. 5

Performance Analysis

• On Weibo, the heatmaps show strong class discrimination for bootstrapped representations but messier cross-modal-consistency representations; on GossipCop, BMR’s t-SNE features have fewer outliers.The heatmaps compare paired cosine similarities between 64-dimensional final-classifier and single-view representations; the t-SNE plots compare BMR, SpotFake, and CAFE.Source: Figure 5 · p. 5, Performance Analysis · p. 6, Figure 6 · p. 6

Experimental Setups

• Experiments use Weibo, GossipCop, and Weibo-21, average the best performance over five initializations per dataset, and provide code and further results in the supplement.BERT and MAE are kept frozen; GossipCop uses a 0.80 threshold, while Weibo and Weibo-21 use 0.50.Source: Experimental Setups · p. 5, Experiments · p. 5

Performance Analysis

• On Weibo-21, BMR leads three open-sourced methods by 4.2% in overall accuracy and 3.2% in fake-news recall.Across Weibo, GossipCop, and Weibo-21, BMR outperforms the compared methods; it is 2.5% ahead of CMC on Weibo.Source: Table 1 · p. 6, Performance Analysis · p. 6
• BMR requires fewer trainable parameters than EANN and SpotFake, while CAFE has lower computational complexity but weaker performance.Training BMR for 50 epochs takes around 12 hours on one NVIDIA RTX 3090 GPU.Source: Table 2 · p. 6, Performance Analysis · p. 6

Ablation Study

• On GossipCop, nearly 50% of images are celebrity faces that are hard to distinguish, and the authors infer that detection relies heavily on text while BMR still beats compared methods in accuracy.Testing accuracy plateaus rather than falling over 50 epochs, which the authors take as evidence of no overfitting.Source: Ablation Study · pp. 6–7, Figure 7 · p. 7
• Bootstrapping three unimodal views without fusion averages 88.6% accuracy, 3.2% below BMR; text alone reaches 87.0%, while prioritizing consistency is not beneficial.Removing cross-modal consistency learning drops performance by 1.3%; consistency-only detection has accuracy below 0.6, so the authors describe consistency as an assistant rather than a critical role.Source: Ablation Study · p. 7, Table 4 · p. 7
• Removing single-view prediction drops performance by 2.3%, and omitting explicit task-based reweighting decreases accuracy by 2.9%.The authors characterize single-view prediction as a useful bias and reweighting as protection against misleading information.Source: Ablation Study · p. 7
• Replacing MAE with ViT lowers overall accuracy by 0.8%; using iMMoE instead of separate ViT blocks raises results by 1.3%, while BMR’s internal MMoE improvement contributes 1.1%.Many learned reweighing functions on GossipCop and Weibo exhibit an inverted V shape.Source: Ablation Study · p. 7

• BMR generates multi-view representations, captures their individual importance, and optimizes the fused features.Source: Conclusions · p. 7
• Single-view prediction and cross-modal consistency learning disentangle unimodal and multimodal information for adaptive reweighting and bootstrapping.Source: Conclusions · p. 7
• Experiments show that BMR outperforms state-of-the-art multimodal fake news detection schemes on popular datasets.Source: Conclusions · p. 7