Source-linked AI summary
Customized large language models can outperform Community Notes in correcting misinformation
Xinyi Zhou, Ashish Sharma, Amy X. Zhang, Tim Althoff
TL;DR
Misinformation correction must address fast-changing, nuanced content, while crowdsourced systems struggle to provide timely, scalable coverage. MUSE combines trust-aware evidence retrieval with multimodal reasoning; across real-world posts it earns higher expert-rated quality than highly rated Community Notes and improves misinformation recognition.
Problem
Misinformation correction must handle multimodal, context-dependent claims and emerging events, while timely and scalable coverage remains difficult for systems such as Community Notes.
Source: Introduction · pp. 1–2Method
MUSE augments LLMs with trust-aware retrieval of up-to-date evidence and multimodal reasoning to identify misleading content and explain assessments with references.
Source: Introduction · p. 2, Approach · p. 3Results
Across diverse real-world posts, MUSE achieved an approximately 29% higher expert-rated response-quality score than highly rated Community Notes and improved participants’ misinformation recognition by approximately 10%.
Source: Introduction · p. 2Takeaways & Limitations
MUSE provides a methodological and evaluative framework for timely, scalable, and trustworthy correction, with high-quality responses across varied posts, including those not previously fact-checked online.
Source: Introduction · p. 2, Discussion · p. 7MUSE supports English text and images but not video; as a retrieval-based system, it depends on relevant, credible online evidence and may lack enough evidence in information voids.
Source: Discussion · p. 8
Abstract
from arXiv · showhide
Addressing misinformation in real-world settings is challenging: content is often multimodal; factuality judgments are nuanced and context-dependent; new events emerge rapidly across domains; corrections must be timely, trustworthy, and politically impartial; and multidimensional, multistakeholder frameworks remain lacking. Crowdsourced fact-checking systems such as Community Notes have gained broad adoption, but timely, scalable coverage remains difficult. We introduce MUSE, which augments large language models (LLMs) with trust-aware retrieval of up-to-date evidence and task-specific multimodal reasoning. Given a piece of content, MUSE identifies whether and which parts may be false or misleading and provides explanations grounded in credible references. We also develop an evaluation framework that assesses expert-rated response quality---including identification accuracy, explanation factuality, and the relevance and credibility of supporting references---as well as user perceptions. Across social media posts spanning modalities, domains, political leanings, misinformation tactics, and popularity, MUSE consistently produces high-quality responses, including for content not previously fact-checked online, and outperforms even highly rated Community Notes by 29%. It also improves participants' recognition of misinformation by 10%. Our work establishes a general methodological and evaluative framework for timely, scalable, and trustworthy correction of misinformation.
Significance Statement
MUSE aims to make misinformation correction timely, scalable, trustworthy, and auditable by grounding multimodal assessments in credible references.
Significance Statement
- MUSE identifies potentially misleading claims in multimodal content and explains its assessments with evidence-grounded references from credible publishers.The references make verification auditable.Source: Significance Statement · p. 1
- The framework targets correction of pervasive misinformation with serious implications for democratic processes, economic stability, public health, and social cohesion.Evaluations with fact-checking experts and laypeople across diverse real-world content demonstrate its effectiveness.Source: Significance Statement · p. 1
- The work presents MUSE as a framework for timely, scalable, and trustworthy misinformation correction.Its assessments use credible references.Source: Significance Statement · p. 1
Introduction
Misinformation correction must handle nuanced, fast-changing multimodal content while remaining timely, explanatory, trustworthy, and impartial. MUSE addresses this challenge with evidence retrieval and multimodal reasoning, and is evaluated with experts and users.
Introduction
- Misinformation threatens democratic processes, economic stability, public health, and social cohesion, while social media can impede truth discernment through rapid spread and habitual sharing.Corrections must address multimodal posts, partial accuracy, omitted context, spurious causal framing, fast-emerging events, and demands for timeliness, explanation, evidence, trust, and political impartiality.Source: Introduction · pp. 1–2
- In a February 2023 snapshot, 88% of suspicious posts lacked any response and 93% lacked a highly rated response within the first hour.Community Notes can be trusted and helpful, but these benefits do not ensure timely or comprehensive coverage.Source: Introduction · p. 2
- Prior LLM fact-checking systems often cover only one workflow stage or operate under restrictions such as text-only inputs or a single domain [22–29].Static benchmarks and automated grading also cannot establish performance on emerging events or the factuality and source quality of complete corrections [30, 12].Source: Introduction · p. 2
- MUSE achieved an approximately 29% higher expert-rated response-quality score than highly rated Community Notes and improved participants’ misinformation recognition by approximately 10%.The evaluation spans real posts across modalities, domains, political viewpoints, misinformation tactics, and popularity, including posts without a prior online fact-check.Source: Introduction · p. 2
- MUSE combines trust-aware retrieval of current evidence with task-specific multimodal reasoning and was evaluated through expert assessment and a 988-participant study of user perceptions.Experts rated 13 dimensions, including identification accuracy, explanation factuality, reference relevance and credibility, and overall response quality.Source: Introduction · p. 2
Approach
MUSE generates corrections for text and image-containing posts by combining an LLM, credibility-aware evidence retrieval, and multimodal processing. It filters and prioritizes evidence before generating an explanation with reference links.
Approach
- MUSE is an end-to-end system that identifies accurate and inaccurate claims in content with text, with or without visuals, explains its judgments, and provides reference links.Its three interconnected modules are an LLM for response generation, a hierarchical credibility-aware web retriever, and a multimodal integrator.Source: Approach · p. 3
- For text-only posts, MUSE generates search queries, retrieves and filters relevant pages, prioritizes credible publishers, extracts evidence, and passes evidence with source links to an LLM for response generation.Publisher priority uses factuality and bias ratings; evidence may refute false claims or provide context for input claims.Source: Approach · p. 3
- Targeted queries decompose multi-claim posts and reduce irrelevant retrieval; low-relevance pages increase hallucinations, while evidence collection stops after two refuting pages or exhaustion of credible pages.Stopping limits processing of large volumes of retrieved page content.Source: Approach · p. 3
- For text-and-image posts, MUSE creates informative image captions with public-figure identification and OCR, then uses multimodal web search and relevance scoring to filter pages.The design seeks to balance privacy considerations with correction effectiveness; captions can retain book titles needed for verification.Source: Approach · p. 3
Experiments
The experiments compare MUSE with Community Notes and model variants using expert ratings of corrections and a separate study of user perceptions. The evaluation includes posts with varied helpfulness ratings and measures belief, sharing intent, and trust.
Experiments
- The evaluation set contained 247 X posts from February 2021 to February 2023, each with at least one high- and one average-helpfulness Community Notes response.High helpfulness was defined as a score of at least 0.35; expert annotations classified 48% of posts as containing both accurate and inaccurate or potentially misleading claims, 46% as inaccurate or potentially misleading, 3% as fully accurate, and 3% as unverifiable.Source: Experiments · pp. 3–4
- Comparisons included matched-time MUSE responses, a version restricted to pages published before each post, the foundation LLM alone, and image-post variants disabling vision or retrieval.Time matching limited retrieved pages to those published no later than 30 minutes before the corresponding Community Notes response.Source: Experiments · p. 4
- Blinded experts rated responses across 13 criteria covering identification and explanation, generated-text quality, references, and overall correction quality on a 10-point scale.Agreement was substantial for reference reachability, moderate for several criteria, and fair for the other kappa-scored criteria.Source: Experiments · p. 5
- A study of 988 U.S. participants measured belief, intention to share, and trustworthiness ratings after participants saw a post and response.Participants were randomly assigned to no source disclosure or disclosure that the correction was human-written or AI-generated.Source: Experiments · p. 5
Results
MUSE outperformed strong Community Notes and GPT-4 baselines in overall response quality, with gains across identification, explanations, text, and references. Its performance remained strong across content types and evaluation settings, and its corrections improved participants’ recognition of misinformation.
Results
- MUSE scored 8.1/10 in mean response quality, 29% above high-helpfulness lay responses (6.3), 37% above GPT-4 (5.9), and 56% above average-helpfulness lay responses (5.2).Lay responses were written an average of 14 hours after posting; MUSE used web pages published before the post.Source: Results · p. 5
- MUSE explicitly identified and explained inaccuracies in 89% of responses; 91% had at least one correct identification and no mistakes, and 70% had fully accurate explanations.MUSE identified all inaccuracies in 61% of responses, versus 38% for GPT-4, 26% for high-helpfulness notes, and 17% for average-helpfulness notes.Source: Results · p. 5
- MUSE’s mean relevance score was 8.7, 18% above GPT-4, 21% above high-helpfulness notes, and 43% above average-helpfulness notes.Its text was completely factual in 74% of cases, versus 59% for high-helpfulness notes and 45% for GPT-4; it also improved relevance and factuality over GPT-4 without reducing fluency, coherence, or toxicity safety.Source: Results · pp. 5–6
- Nearly 100% of MUSE’s reference links were reachable and 96% were relevant; 49% of GPT-4 links returned page-not-found errors, and 76% of its reachable links were relevant.MUSE’s references were also more credible than those in high-helpfulness lay responses.Source: Results · p. 6
- MUSE’s mean response quality remained 8.1±2.0 on posts both before and after the assumed September 2021 GPT-4 training cutoff, and it retained an 8.0 mean after excluding posts with related fact-checks.For the posts without cited fact-checks, MUSE was at least 28% higher quality than GPT-4 and high-helpfulness lay responses.Source: Results · p. 6
- MUSE’s mean quality was 8.3 for both liberal- and conservative-leaning misinformation, and it outperformed baselines across modalities, domains, tactics, and popularity levels.Its advantage was at least 21% for text-only and multimodal posts, at least 19% across domains and tactics, and at least 26% for both political-leaning groups.Source: Results · p. 6
- Removing retrieval lowered MUSE’s overall response quality by 25%, and removing vision lowered it by 33%; quality was similar across simulated correction start times.Mean response quality was 8.1±2.0 whether correction began immediately, at 13 hours, or at 16 hours.Source: Results · p. 7
- MUSE-generated corrections increased participants’ correct belief that misinformation was misleading by 10%, while no significant changes in belief were observed for GPT-4 or lay responses.MUSE’s trustworthiness score was 4.9, 8% higher than average-helpfulness lay responses; withholding source disclosure increased correct belief by 8%, primarily for less effective corrections.Source: Results · p. 7
Discussion
The authors present MUSE as a training-free framework for timely, transparent misinformation correction, with strong expert-rated quality and improved user recognition. They also identify boundaries in supported languages, platform coverage, evidence availability, and evaluation scope.
Discussion
- MUSE is proposed as a practical end-to-end framework that produces high-quality corrections across modalities, political leanings, domains, tactics, popularity levels, and claims not previously fact-checked online.Its pipeline is described as simple, training-free, and intended to facilitate deployment and maintenance.Source: Discussion · p. 7
- MUSE corrections can be generated within minutes, averaging two minutes on a basic M1 CPU with 16 GB of memory, and improve misinformation recognition regardless of source disclosure.Transparent evidence links let users verify responses; the system also signals insufficient evidence, provides accuracy nudges, and expresses uncertainty when appropriate.Source: Discussion · p. 7
- MUSE costs approximately 0.2 USD per post, and replacing GPT-4 with Llama 3 70B produced comparable qualitative-analysis performance with other implementation details fixed.The authors suggest this indicates MUSE is not tied to a specific foundation model and may support lower-cost deployment.Source: Discussion · pp. 7–8
- Vanilla GPT-5.5 had a 43% reference-hallucination rate, versus 2% for MUSE; only 70% of its links were reachable and 81% of reachable links were relevant and supportive.The authors caution that comparisons among model-based MUSE variants are not strictly fair because their training cutoffs may include evaluation posts, corrections, or Community Notes.Source: Discussion · p. 8
- MUSE supports English text and images, not video, and was evaluated on X alone; reliable verification may be constrained when relevant, credible online evidence is absent.The authors did not systematically evaluate information-void detection or abstention reliability; the modest, skewed Community Notes sample and immediate self-reported user outcomes also limit generalizability.Source: Discussion · p. 8
Data Availability
Study data and code are available through the project repository, subject to privacy-related release constraints.
Data Availability
- The study’s data and code are available in the MUSE GitHub repository; only tweet IDs are released to comply with X’s Terms of Service, and experts’ names are anonymized.The repository is https://github.com/Social-Futures-Lab/MUSE.Source: Data Availability · p. 15