Source-linked AI summary
Beyond Truth Discovery: A Two-Stage Framework to Assess the Severity of False Claim during Disasters
Ruichen Yao, Tejna Dasari, Gulshat Baispay, Aizhan Zaurbek, Yifan Liu, Yaokun Liu, Zelin Li, Dong Wang
TL;DR
Existing research mainly detects false information at the post level, offering limited insight into individual false claims and their severity during disasters. This paper introduces a two-stage claim-level framework and benchmark, finding that LLMs—especially with in-context learning—align more strongly with human severity judgments than traditional supervised models.
Problem
Existing approaches mainly assess whether entire social media posts are false, providing limited analysis of individual false claims and their severity during disasters.
Method
The paper extracts and verifies false claims from multimodal disaster-related social media, then assesses severity through human-rated believability and harmfulness in a claim-level benchmark.
Results
In-context learning consistently achieves the strongest alignment with human judgments, while LLMs substantially outperform traditional supervised models.
Takeaways & Limitations
Human-labeled examples and shared decision criteria provide a stronger alignment signal for severity assessment than additional reasoning alone.
Takeaways & Limitations
Validation confirms extraction correctness but not recall, so the framework may miss additional valid false claims.
Abstract
from arXiv · showhide
False information spreads rapidly on social media during disasters and can undermine emergency response efforts, public trust, and crisis communication. Existing research primarily focuses on determining whether social media posts contain false information, but provides limited insight into the specific false claims embedded within posts and the severity of individual false claims. To address the limitations, we propose a two-stage framework to assess the severity of false claims during disasters. In the first stage, we develop a false claim extraction agent that identifies false claims from multimodal social media posts containing text, images, videos, and links. A subsequent verification step validates extracted claims with supporting evidence. In the second stage, we define false claim severity as the combination of two complementary dimensions: believability, which determines the likelihood that a claim will be believed, and harmfulness, which captures the potential consequences if it is believed. Human annotators assess both dimensions to construct a claim-level severity benchmark using false claims extracted from Reddit posts related to hurricanes and wildfires. Building upon this benchmark, we investigate false claim severity assessment as a human-AI alignment problem, evaluating whether models can reproduce human judgments under a shared evaluation rubric rather than merely predicting severity labels. Experiments on the benchmark show that traditional supervised models exhibit limited alignment with human judgments, whereas Large Language Models (LLMs) achieve substantially stronger performance. Among the evaluated strategies, in-context learning consistently achieves the strongest alignment with human judgments, highlighting the importance of human examples and shared decision criteria for severity assessment.
1 Introduction
The paper argues that disaster-related false information requires claim-level analysis and severity assessment beyond post-level truth detection. It proposes a two-stage framework that extracts and verifies false claims, then evaluates their believability and harmfulness against human judgments.
- Motivation: False information spreads rapidly during disasters, potentially causing public panic, disrupting rescue operations, and undermining trust in emergency response agencies.
- Research gaps: Existing truth-discovery approaches primarily determine whether entire social media posts contain false information rather than identifying individual false claims.
- Research gaps: Prior work also lacks severity evaluation, although false claims differ in likely audience belief and potential social consequences.
- Two-stage framework: The first stage uses a Multimodal Large Language Model to extract false claims from textual and visual content and self-verify them against external supporting evidence.
- Two-stage framework: The second stage defines severity through believability and harmfulness, using expert-designed guidelines and human annotations to create a claim-level benchmark for human-AI alignment.
2 Related Works
Prior work documents the spread of disaster-related false information and develops detection and verification approaches, but rarely assesses the severity of individual false claims. This gap motivates a systematic framework for claim-level severity assessment.
- Disaster False Information: Social media is a primary channel for obtaining and spreading information during natural disasters, where false information can mislead the public.Prior research highlights the rapid spread of false information during natural disasters and terrorist attacks.
- Disaster False Information: Studies of the 2012 hurricane season emphasize that verified accounts can help curb false information because the public relies on their credible information.Verified accounts contribute by providing credible information that debunks false claims.
- Disaster False Information: Machine learning models have been proposed to detect false information on social media during natural disasters.
- Severity Assessment Gap: Existing veracity research primarily identifies and verifies false information, while only a small body of work examines its risks, harms, or severity.Prior studies consider consequences such as safety risks, panic, and disease exposure during humanitarian crises.
- Severity Assessment Gap: Prior studies do not provide a systematic framework for assessing the severity of individual false claims, motivating the proposed two-stage framework.Existing work mainly examines consequences at conceptual or event levels rather than systematically assessing individual claims.
3 False Claim Severity Assessment Framework
The framework uses two stages: extracting and verifying individual false claims from multimodal social media posts, then assessing each claim’s severity through believability and harmfulness. An MLLM-based agent aggregates multimodal evidence and self-verifies candidate claims, while human judgments define claim-level severity.
- Stage 1: False Claim Extraction: The first stage extracts individual false claims from multimodal social media posts and verifies them with supporting evidence.The framework uses an MLLM-based extraction agent and three independent annotators to improve claim quality and reliability.
- Stage 1: False Claim Extraction: The extraction agent aggregates text, images, videos, and external links, using 10 representative video frames to capture video evidence.Candidate frames are uniformly sampled and encoded with the CLIP vision encoder before semantic keyframe selection.
- Stage 1: False Claim Extraction: The agent identifies assertions, retrieves reliable external evidence through claim-specific searches, and applies self-verification to reduce retrieval and reasoning errors.Evidence sources include government reports, fact-checking websites, and established news; self-verification reviews preliminary candidate claims with their evidence.
- Stage 1: False Claim Extraction: The extraction agent achieved an accuracy of 0.98 on the hurricane subset and 0.99 on the wildfire subset against majority-voted human annotations.Three annotators reviewed 600 extracted claims: 300 from the hurricane dataset and 300 from the wildfire dataset.
- Stage 2: Severity Assessment: The second stage assesses severity as a human-centered combination of believability and harmfulness rather than as a single-dimensional construct.Believability captures how credible and trustworthy a claim appears, while harmfulness captures the potential consequences if it is accepted; believability is represented as low or high.
4 Human-AI Alignment
The study evaluates human-AI alignment on 600 manually annotated false claims using a shared rubric for believability, harmfulness, and overall severity. Traditional supervised models show limited alignment, while LLMs perform substantially better, with in-context assessment consistently strongest but still imperfect.
- Evaluation Setup: Experiments use majority-voted annotations from 600 false claims as the gold standard, with models independently assessing believability and harmfulness under the human annotation rubric.Evaluations cover Hurricane-only, Wildfire-only, and Cross-disaster settings.
- Traditional Supervised Models: Traditional supervised models achieve moderate performance on believability and harmfulness but decline on combined four-category severity assessment.The combined task is more complex than predicting either dimension in isolation.
- Traditional Supervised Models: Low F1 and Cohen’s Kappa scores indicate limited alignment with human judgments, especially for minority severity categories.Accuracy appears to be driven largely by majority classes rather than uniformly distributed performance.
- LLM-Based Models: LLM-based approaches substantially outperform traditional supervised models across hurricane and wildfire datasets, particularly on F1 and Cohen’s Kappa.The stronger scores indicate improved alignment across believability, harmfulness, and overall severity.
- LLM-Based Models: In-Context Assessment consistently achieves the strongest performance for both GPT and Qwen, especially on overall severity assessment.Human-annotated examples provide a stronger alignment signal than free-form reasoning alone.
- LLM-Based Models: A noticeable gap remains between LLM predictions and majority-voted human annotations, showing that human severity judgments are not fully reproduced.The remaining gap applies to assessments of believability and harmfulness.
5 Limitations
The study validates the accuracy of extracted false claims but cannot assess extraction recall, and its small annotated dataset may affect supervised-model performance and the observed performance gap.
- Extraction recall: Human verification confirms extracted false claims are highly accurate but does not determine whether additional valid claims were missed.Manually identifying all possible false claims in multimodal social media posts is prohibitively expensive.
- Annotated data: The relatively small annotated dataset may hinder supervised models’ ability to learn false-claim severity patterns, so the performance gap requires cautious interpretation.Under the current data setting, LLM-based approaches show stronger alignment with human judgments; larger and more diverse datasets could improve evaluation.
6 Conclusion
The paper presents a two-stage framework that advances disaster-related false-information analysis from post-level detection to claim-level severity assessment. In-context learning performs best, indicating that human examples and shared decision criteria strengthen human-AI alignment.
- Framework: The framework introduces a two-stage approach for assessing the severity of false claims in disaster-related social media.It moves beyond prior post-level false-information detection toward claim-level analysis.
- Framework: A multimodal false claim extraction agent identifies individual false claims from social media posts.The agent supports claim-level analysis across multimodal content.
- Human-AI alignment: In-context learning consistently produces the best results among the evaluated strategies.This finding concerns alignment with human judgments of false claim severity.
- Human-AI alignment: Human-labeled examples and shared decision criteria provide a stronger alignment signal than additional reasoning alone.The conclusion emphasizes the role of shared evaluation guidance in human-AI alignment.
- Future research: The work provides a foundation for research on false claim severity assessment, human-AI alignment, and scalable moderation of disaster-related false information.These are identified as directions supported by the proposed framework and findings.