Detecting and Explaining Fake News Short Videos with Multimodal Content and Real-World Evidence

1 Introduction

• Short-video platforms are major channels for daily news and fake-news dissemination, while video misinformation spreads faster and seems more trustworthy than text- or image-based versions.Source: §1 · p. 1
• Random sampling may omit brief veracity-relevant cues or temporal context, and 93.4% of analyzed FakeSV videos exhibit at least one of four frame-selection challenges.Source: §1 · p. 2
• Prior FNVDE methods focus on either multimodal content analysis or external evidence retrieval, while retrieved web evidence may be noisy or misleading.Source: §1 · p. 2
• CEI separates content-based analysis from evidence-based verification, then fuses both checkers’ outputs with a lightweight judge; query regeneration and text attention address retrieval noise.Source: §1 · pp. 2–3
• NVKE selects keyframes by tracking chronological changes in visual-text similarity to preserve content changes and temporal context while reducing redundancy.Source: §1 · p. 3
• Acc. gains over the second-best results are 0.0071 on FakeSV and 0.0234 on FakeTT; Macro F1 gains are 0.0002 and 0.0190, respectively.The experiments compare NVKE-CEI with thirteen baselines.Source: §1 · p. 3

2 NVKE-CEI

• NVKE-CEI combines keyframe extraction with a framework that uses content and evidence information for veracity prediction and content-grounded explanation.Source: §2 · p. 3

2.1 Problem Setting

• FNVDE predicts a news video’s real-or-fake label and generates an explanation grounded in its news content.Source: §2.1 · p. 3

2.2 NVKE: News Video Keyframes Extraction

• NVKE embeds frames and OCR text with CLIP, combines consecutive visual and text similarities, then ranks change points and selects their boundary frames as keyframes.Source: §2.2 · pp. 3–4

2.3 CEI: FNVDE Framework with Content and Evidence Information

• The two LLM checkers use structured outputs to provide reasoning and confidence scores, with the evidence-based checker’s output varying by dialogue stage.Source: §2.3 · p. 4

2.3.1 Content-based LLM

• The content-based checker analyzes cross-modal consistency, technical authenticity, content logic, and propagation intent, returning reasoning and a fake-class confidence score.Source: §2.3.1 · p. 4

2.3.2 Evidence-based LLM

• The evidence-based checker iteratively reasons about the video, regenerates queries for specificity and relevance, retrieves web evidence, and uses it to predict veracity.Source: §2.3.2 · p. 5

2.4 Lightweight Judge Model

• The judge attends over text features, combines text and video representations, and injects the two checkers’ averaged fake-class log-odds as a prior into its fake-class logit.Source: §2.4 · p. 6

3.1 Experiment Settings

• Experiments use FakeSV and FakeTT, evaluating detection with Acc. and Macro F1 against thirteen baselines, including zero-shot LLM and supervised methods.The experiment section also outlines additional detection, ablation, and explainability evaluations.Source: §3.1 · pp. 6–7, §3 · p. 6

3.2 Experiments Results

• 0.0071 (resp. 0.0234) in Acc. and 0.0002 (resp. 0.0190) in Macro F1 are NVKE-CEI’s gains over second-best results on FakeSV (resp. FakeTT).The study reports the optimal trade-off between time efficiency and performance among baselines.Source: Table 1 · p. 7, §3.2 · p. 7
• Ablations show that NVKE improves performance, while removing either fact-checking LLM reduces results on both datasets; judge components are also essential.Removing NVKE reduces Acc. and Macro F1 on both datasets, and removing the evidence-based LLM causes the largest degradation among LLM-component ablations.Source: Table 2 · p. 7, §3.2 · pp. 7–8

3.3 Model Explainability

• On FakeTT, NVKE-CEI scores 4.96 in I, 4.90 in S, and 4.87 in P, achieving the highest scores across all five explanation-quality metrics.Its I score exceeds ExMRD by 0.04; its S and P scores exceed ReAct by 0.09 and 0.15, respectively.Source: Table 3 · p. 8, §3.3 · p. 8
• The judge prefers the evidence-based LLM when strong evidence exists during conflicts, and defaults to the content-based LLM when evidence is insufficient or contradictory.The prediction distribution is presented for Real and Fake labels.Source: Table 4 · p. 8, §3.3 · p. 8

FNVDE

• Early FNVDE studies focused on extracting and integrating multimodal features, while later work added social information, neighboring event videos, or creative-process modeling.Recent LLM-based approaches use sampled frames, keyframes, or visual descriptions, but may omit brief cues or temporal dependencies; some rely solely on internal LLM knowledge.Source: §4 · pp. 8–9

5 Conclusion

• NVKE-CEI combines keyframe extraction with content and evidence information, achieving SOTA detection performance on two datasets with thirteen baselines while providing content-grounded explanations.The conclusion describes the keyframe strategy as addressing possible loss of veracity-relevant cues or temporal context.Source: §5 · p. 9

Limitations

• NVKE may miss gradually evolving veracity-relevant cues when frame-to-frame similarity remains high.Future work will explore temporal modeling and fine-grained manipulation detection.Source: Limitations · p. 9
• When both LLMs converge on the same incorrect conclusion, their agreement may leave the judge without a sufficiently strong contradictory signal to correct the error.The authors propose external validation or diverse reasoning strategies as future directions.Source: Limitations · p. 9
• The framework relies on proprietary GPT-4o APIs and live web search, making latency, cost, retrieved evidence, and outputs variable with service versions and availability.Processing one batch of 15 samples takes 29.6 seconds under the experimental setup, which may hinder practical deployment.Source: Limitations · p. 9
• Explanation quality is evaluated only on FakeTT using one LLM evaluator and no human judgments, so reported G-Eval scores may reflect evaluator-specific preferences.The evaluation does not establish whether generated explanations are faithful to the final judge prediction.Source: Limitations · p. 9

A Pilot Study on FakeSV Dataset

• The pilot sampled one frame per second, stitched each four consecutive frames, and asked GPT-4o-mini to label four types of frame-selection challenge.The labels were semantic redundancy, keyframe sparsity, cross-frame reasoning gap, and temporal discontinuity.Source: A Pilot Study on FakeSV Dataset · pp. 12–13
• 93.4% of analyzed FakeSV videos had at least one frame-selection challenge; semantic redundancy was more frequent in fake videos, while keyframe sparsity and cross-frame reasoning were more frequent in real videos.The pilot repeated five times with negligible variance (mean 93.4%, standard deviation 0.12%).Source: Table 5 · p. 13, A Pilot Study on FakeSV Dataset · p. 13

B.1 More Details of the Datasets and Evaluation Metrics

• FakeSV is a Chinese dataset sourced from Douyin and Kuaishou, while FakeTT is an English dataset sourced from TikTok; evaluation uses accuracy and Macro F1.Uncertain LLM predictions are treated as incorrect in evaluation.Source: §B.1 · p. 13

B.2 Baselines Implementation Details

• Zero-shot LLM baselines use a Hu et al. (2024a)-based prompt, supervised settings follow original studies, and SVFEND uses video content alone on FakeTT.FakeTT does not include social context such as user profiles or comment interactions.Source: §B.2 · pp. 13–14

B.3 NVKE-CEI Implementation Details

• NVKE uses α = 0.5 and K = 4; RAG retrieval is limited to five results per query and excludes information published after fact-checking.The authors selected K = 4 because gains beyond four were negligible while API processing time increased significantly.Source: §B.3 · p. 14

B.4 Latency and Performance Report and Discussion

• Among five evaluated API-based methods, NVKE-CEI achieves the highest accuracy and second-lowest latency; its two LLM checkers process batches of 15 videos in about 30 seconds.GPT-4o has the lowest time cost, while NVKE-CEI outperforms it in accuracy.Source: §B.4 · pp. 14–15, Table 8 · p. 15

• An additional experiment tests NVKE-CEI with Qwen3-vl-plus, with detailed results presented in Table 6.Source: C More Experiments on Different LLM… · p. 15

D More Experiments on NVKE and Case Study

• NVKE-CEI outperforms ReAct+KNN in the reported comparison, and frame information is decisive for detection in the illustrative case study.Source: D More Experiments on NVKE and Case… · p. 15

E.1 Detail Settings of G-Eval

• Explanation evaluation uses GPT-4o at temperature 0, adapted from G-Eval, with ratings on a 1–5 Likert scale.The evaluation prompt defines the criteria used for rating explanations.Source: §E.1 · p. 15

E.2 Qualitative Analysis on Explainability

• The content-based checker flags OCR text, computer-generated visual effects, and sensational hashtags as red flags against the asteroid claim.Its reasoning cites astronomical implausibility, imagery inconsistent with authentic footage, and the hashtag #scary.Source: §E.2 · p. 15
• The evidence-based checker searches for corroboration and finds only discussion of possible future events, not credible sources confirming the claimed strike.It generates the targeted query “Moon struck by asteroid October 2023.”Source: §E.2 · p. 15
• The judge combines both reasoning streams to predict fake, and both streams contribute to the content-grounded explanation.Source: §E.2 · p. 15

F Error Analysis

• In both dataset error cases, the two checkers independently label real videos fake, leaving the fusion module without corrective signals and producing wrong final predictions.The analysis notes that dual perspectives may amplify errors in these edge cases.Source: F Error Analysis · pp. 15–16
• The content-based checker can mistake rare but genuine events for fakes, including a construction accident and the real date February 29, 2024.The authors connect these errors to skepticism toward unusual events and limited conventional visual documentation.Source: F Error Analysis · pp. 16–17
• The evidence-based checker can treat missing search results as proof of fabrication, overlooking limited English reporting, local terminology, and temporal restrictions.The ReAct process did not initiate further iterations before deciding it had enough evidence.Source: F Error Analysis · p. 17

H More Related Work

• Related work describes noisy retrieved evidence and imperfect query generation as limitations, while prior video-explanation methods may offer only suspicious-region highlighting or fake-video explanations.The cited work also notes that MRGT requires labeled news data and human-generated explanations for training.Source: H More Related Work · pp. 17, 18, 20

G.1 Prompt for Pilot Study

• The pilot prompt classifies sampling problems as keyframe sparsity, temporal discontinuity, semantic redundancy, cross-frame reasoning failure, or none of these.It asks for analysis of all video frames and labels affecting detection accuracy.Source: §G.1 · p. 17, H More Related Work · p. 19

G.2 Prompt for Evidence-based LLM

• The evidence-checker prompt directs factual, spatiotemporal, source-credibility, and propagation-intent analysis, including whether web search is needed.It requests a veracity confidence score and, optionally, search queries.Source: §G.2 · p. 17, H More Related Work · p. 19

G.3 Prompt for Content-based LLM

• The content-checker prompt evaluates cross-modal consistency, technical authenticity, content logic, and propagation intent.It treats technical manipulation as a strong misinformation indicator, while requiring anomalies to be considered alongside other factors.Source: §G.3 · p. 17, H More Related Work · p. 19

G.4 Prompt for Query Regeneration

• The query-regeneration prompt assesses relevance, specificity, information coverage, and fact-checking value before retaining or refining search terms.Source: §G.4 · p. 17, H More Related Work · p. 20