Source-linked AI summary

Fake News Theories: Harnessing Disciplinary Insights for Computational Modeling, Detection, and Explanation

Zhaoyang Cao, Miriam Metzger, Reza Zafarani

arXiv:2609.30427v1cs.LGcs.CY

TL;DR

Automated fake-news detection faces a gap in interpretability and connection to established theories of persuasion and credibility. This paper reviews cross-disciplinary theories and translates them into computational features; experiments find predictive, interpretable signals whose importance and effectiveness vary across datasets.

  • Problem

    Many automated fake-news detectors provide limited interpretability, despite the need for evidence that can support decisions and help assess model bias and fairness.

    Source: §I · p. 1
  • Method

    The paper reviews social-science theories and translates their concepts into measurable features using statistical and machine-learning methods, including large language models.

    Source: §I · p. 2, §V · p. 9
  • Results

    Across benchmark datasets, theory-derived features contain meaningful diagnostic information for distinguishing fake from real news, although their relative importance and effectiveness vary by dataset.

    Source: §VI · p. 9
  • Takeaways & Limitations

    The framework offers a complementary trade-off between predictive performance and theoretical transparency through compact, human-interpretable signals.

    Source: §VI · p. 9

    Datasets differ in topic, class distribution, and source composition, and pretrained or large language model representations may be sensitive to prompting and domain shift.

    Source: §V · p. 9

Abstract

from arXiv · show

Disinformation research has produced increasingly accurate automated fake-news detectors, but many systems remain difficult to interpret and are weakly connected to established theories of persuasion, credibility, and human judgment. In this paper, we develop a theory-informed computational framework that translates cross-disciplinary theories of fake news into measurable features for automated detection and explanation through statistical techniques and large language models. To that end, we conduct a structured cross-disciplinary review of theories from social sciences, psychology, economics, among other disciplines that reveal how fake news persuades and spreads, thereby establishing a broad theoretical foundation for computational modeling. Experiments on benchmark datasets show that theory-derived features are predictive and provide interpretable, theory-referenced diagnostic signals. Multi-feature models generally outperform individual features, although gains among the strongest small feature combinations are modest. Our work highlights the value of interdisciplinary perspectives in building robust and interpretable fake news detection systems, advancing the foundation for human-centered approaches in combating disinformation.

I. INTRODUCTION

Automated fake-news detection is needed to handle online information at scale, but systems often lack interpretable evidence. The paper motivates a broader theory-based foundation, organizing relevant theories around news content, sources, and networks.

I. INTRODUCTION

  • Online information’s volume, velocity, and variety make reliable manual detection challenging, while rapidly spreading disinformation can magnify even small errors.The paper argues automated tools are essential because manual detection is impractical in today’s digital information ecosystem.Source: §I · p. 1
  • Many detection systems prioritize prediction but offer limited explanations, reducing their utility for journalists, policymakers, and fact-checkers and complicating bias and fairness assessment.The paper identifies interpretable evidence as useful for justifying decisions and for identifying systematic model biases.Source: §I · p. 1
  • Prior theory-informed studies demonstrate useful psychological, linguistic, and diffusion cues, but address a narrower set of perspectives than a comprehensive account requires.The paper calls for a broader, multi-theoretical perspective to identify diverse features of deceptive content.Source: §I · p. 2

II. THEORETICAL FOUNDATION

  • The review emphasizes attention and persuasion because fake news is described as primarily seeking attention and audience persuasion for financial or political gain.The passage contrasts this emphasis with true news placing more weight on informing citizens.Source: §II · p. 2
  • Theories are organized around content, source, and network factors; network theories are reviewed but not empirically evaluated because benchmark datasets lack propagation traces and inter-source networks.Content-based theories address message features, source theories address credibility, and network theories include consensus, homophily, and propagation dynamics.Source: §II · p. 2

III. THEORY-INFORMED COMPUTATIONAL FRAMEWORK

The framework selects social-science theories that are applicable and testable on data for distinguishing fake from real news, aiming for a diverse and representative set.

III. THEORY-INFORMED COMPUTATIONAL FRAMEWORK

  • The framework relies on theories that are both applicable and data-testable for distinguishing fake from real news.The passage points to Section III-B for the procedures used to build systematic features that capture these theories.Source: §III · p. 3
  • The selected theories are intended to capture diverse dimensions of fake-news dynamics through a representative set.Source: §III · p. 3
  • The framework’s feature-building procedures are described as systematically capturing theories relevant to distinguishing fake from real news.Source: §III · p. 3

A. Problem formulation

The system represents news articles with theory-driven features and learns a supervised mapping from those representations to true or fake labels. It operationalizes selected content- and source-related theories using information available in static articles and publisher data.

A. Problem formulation

B. News Representation and Classification

  • The empirical feature set covers ten content-based representations and a source-related representation when publisher information is available.The paper limits its empirical focus to content and source theories because its datasets lack user interactions and social-diffusion records.Source: B. News Representation and… · p. 3
  • Certainty is derived by comparing certainty- and uncertainty-word frequencies; scarcity receives an LLM-guided binary label, and negativity uses VADER sentiment.The article is labeled certain when certainty-related words occur more frequently; VADER compound scores range from -1 to +1.Source: B. News Representation and… · p. 4
  • Textual coherence averages entity-transition log-probabilities, while supporting evidence aggregates normalized densities of five in-text signals.The five signals are URLs, quotations, temporal references, attribution phrases, and URLs from traditionally trustworthy domains.Source: B. News Representation and… · p. 4
  • The system measures claim repetition with an LLM, textual complexity with Flesch-Kincaid Reading Ease, and language vagueness with a fine-tuned BERT classifier.The vagueness classifier is trained on sentence ratings from a corpus of privacy policies before being applied to news articles.Source: B. News Representation and… · pp. 4–5
  • The framework also represents logical consistency as the proportion of adjacent sentence pairs classified as entailment.The method uses a pretrained natural-language-inference model to classify sentence pairs as entailment, contradiction, or neutral.Source: B. News Representation and… · p. 4
  • Information gap combines headline punctuation cues with topic-level surprisingness relative to a corpus baseline.When a headline has no trailing punctuation, an LLM scores a restricted set of possible next punctuation marks using a restricted softmax.Source: B. News Representation and… · p. 5
  • Source credibility is encoded from publisher-level reliability annotations, while information gap combines punctuation-driven curiosity and topic surprisingness.The information-gap feature uses a linear fusion of punctuation and surprisingness scores, with default weights of 0.5 each.Source: B. News Representation and… · p. 5

IV. EVALUATION AND RESULTS

The evaluation tests individual theories, cross-dataset stability, relative contributions, and interactions. Its dataset-statistics table is identified, but the supplied passage gives no statistics.

  • The evaluation examines individual-theory significance, cross-dataset stability, relative contributions, and interactions.Source: §IV · p. 5
  • Table III is identified as presenting dataset statistics, but no values are provided here.Source: Table III · p. 6

A. Experimental setup

Experiments use three public fake-news benchmarks and report average AUC and macroaveraged F1. Text preprocessing lowercases the text and removes paragraph separators when present.

A. Experimental setup

B. Baseline Comparison and Predictive Trade-off

The paper compares theory-informed features with TF-IDF using the same classifier, data partitions, and metrics. TF-IDF performs better, while the theory-informed representation trades some predictive performance for compactness and interpretability.

B. Baseline Comparison and Predictive Trade-off

C. Individual importance of theories

On RECOVERY, combining theory-driven features improves predictive performance and interpretability over individual features. Feature analyses identify source credibility and information complexity as prominent, while correlations and SHAP offer complementary views of feature behavior.

C. Individual importance of theories

D. Cross-dataset comparison

Feature importance varies across datasets, with source credibility dominant where publisher metadata is available and several content-based signals contributing more consistently. Dataset-specific factors also shape which theories are most prominent.

D. Cross-dataset comparison

  • 97.2% on GOSSIPCOP and 45.8% on RECOVERY: source credibility is the most decisive signal where publisher metadata is available.It is excluded from the cross-dataset evaluation plot because FAKE AND REAL NEWS lacks reliable publisher venue annotations and its dominance obscures other features.Source: D. Cross-dataset comparison · p. 7
  • Information complexity, logical consistency, coherence, and negativity bias show stable, moderate contributions across datasets.The authors interpret these content-based signals as generalizing beyond domains.Source: D. Cross-dataset comparison · p. 7
  • The scarcity principle is more prominent in FAKE AND REAL NEWS, while feature importance also varies with dataset characteristics such as publisher-level metadata.The authors suggest stronger urgency or limited-availability rhetoric in that dataset.Source: D. Cross-dataset comparison · p. 7

E. Potential Patterns of Fake News Content

Fake news differs from real news across several theory-driven content features, including sentiment, information complexity, linguistic vagueness, and information gaps.

E. Potential Patterns of Fake News Content

  • Fake news has significantly lower sentiment scores and information complexity than real news.The authors associate these patterns with stronger negativity bias and simpler narrative structures with reduced lexical and syntactic diversity.Source: E. Potential Patterns of Fake News… · p. 7
  • Fake news shows greater linguistic vagueness, with more ambiguous or imprecise expressions.The authors suggest such expressions may obscure factual details and mislead judgments of truthfulness.Source: E. Potential Patterns of Fake News… · p. 7
  • Fake news demonstrates a larger information gap, with more uncertainty-inducing headline cues such as question marks, ellipses, exclamation marks, and surprising topics.The authors suggest these cues may arouse curiosity and draw readers’ attention.Source: E. Potential Patterns of Fake News… · p. 7

F. Cross-Theoretical Interaction Analysis

Theory-informed feature combinations can improve predictive performance, but their benefits are modest and vary across datasets. Feature-importance rankings show weak alignment between RECOVERY and FAKE AND REAL NEWS.

F. Cross-Theoretical Interaction Analysis

  • Supporting evidence achieves the highest single-feature AUC (0.742), followed by the confidence heuristic (0.709).A feature can predict labels well alone yet receive lower importance in a joint model when other features capture overlapping or substitutable information.Source: F. Cross-Theoretical Interaction… · p. 8
  • A confidence heuristic, scarcity principle, and supporting evidence achieve AUC 0.745, surpassing any single feature, though combination gains are not uniformly additive.The authors caution that this is not evidence of strong synergistic effects.Source: F. Cross-Theoretical Interaction… · p. 8
  • Feature-importance rankings align weakly between RECOVERY and FAKE AND REAL NEWS: Spearman rank correlation is 0.167 and Kendall Tau correlation is 0.111.The authors note that relative feature importance is affected by dataset-specific factors such as topic domain, publisher diversity, and stylistic conventions.Source: F. Cross-Theoretical Interaction… · pp. 8–9
  • Confidence heuristic + supporting evidence (0.740) and scarcity principle + supporting evidence (0.737) lead pairwise results; information complexity + supporting evidence scores 0.732.The authors describe the pairings as combining assertive language or urgency with verifiable in-text evidence, or structural organization with external attribution.Source: F. Cross-Theoretical Interaction… · p. 8, Figure 6 · p. 9

V. DISCUSSION AND LIMITATIONS

The paper maps social-science theories to computationally measurable news-credibility signals and evaluates their predictive role and directionality. Its conclusions characterize these signals as diagnostic and interpretable, but complementary to full-text representations.

V. DISCUSSION AND LIMITATIONS

  • The framework maps social-science perspectives on persuasion, credibility, language use, and information processing to measurable news-credibility features.It operationalizes theories using lexicon-based measures, statistical machine learning, and large language models to expose predictive roles and directionality beyond latent model states.Source: §V · p. 9
  • Dataset differences in topic, class distribution, and source composition may contribute to dataset-specific feature importance; pretrained or large language model representations may be sensitive to prompting and domain shift.The authors call for robustness evaluation across prompting strategies and application domains, and for extending the framework to propagation-aware datasets.Source: §V · p. 9

VI. CONCLUSION

  • Across benchmark datasets, theory-derived features provide diagnostic information, but their relative importance and effectiveness vary; the framework is complementary to full-text representations rather than intended to outperform them.The authors frame the approach as a trade-off between predictive performance and theoretical transparency.Source: §VI · p. 9
Loading 2609.30427v1…