Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection

Beizhe Hu; Qiang Sheng; Juan Cao; Yuhui Shi; Yang Li; Danding Wang; Peng Qi

Source-linked AI summary

Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection

Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, Peng Qi

arXiv:2309.12247v2Sep 21, 2023cs.CLcs.AIcs.CY

TL;DR

Fake news detection requires diverse clue recognition and real-world knowledge, while the potential role of LLMs remains underexplored. The paper studies GPT-3.5 and develops ARG and rationale-free ARG-D to let SLMs selectively use LLM rationales. Experiments on two real-world datasets show that both methods outperform SLM-only, LLM-only, and combined-model baselines.

Problem
Fake news detection requires diverse clues and profound real-world understanding, but SLMs have knowledge and capability limitations and LLM use in this task remains underexplored.
Method
The paper empirically studies LLM judgments and rationales, then designs ARG to selectively guide SLMs with LLM rationales and ARG-D through distillation without querying LLMs.
Results
ARG and ARG-D outperform existing SLM-only, LLM-only, and combined small–large language model methods on two real-world datasets.
Takeaways & Limitations
Current LLMs may not substitute fine-tuned SLMs for fake news detection but can advise them with instructive rationales.
Takeaways & Limitations
The study does not examine other well-known LLMs because their APIs were unavailable, and its best results remain below oracle voting integration.

Abstract

from arXiv · show

Detecting fake news requires both a delicate sense of diverse clues and a profound understanding of the real-world background, which remains challenging for detectors based on small language models (SLMs) due to their knowledge and capability limitations. Recent advances in large language models (LLMs) have shown remarkable performance in various tasks, but whether and how LLMs could help with fake news detection remains underexplored. In this paper, we investigate the potential of LLMs in fake news detection. First, we conduct an empirical study and find that a sophisticated LLM such as GPT 3.5 could generally expose fake news and provide desirable multi-perspective rationales but still underperforms the basic SLM, fine-tuned BERT. Our subsequent analysis attributes such a gap to the LLM's inability to select and integrate rationales properly to conclude. Based on these findings, we propose that current LLMs may not substitute fine-tuned SLMs in fake news detection but can be a good advisor for SLMs by providing multi-perspective instructive rationales. To instantiate this proposal, we design an adaptive rationale guidance network for fake news detection (ARG), in which SLMs selectively acquire insights on news analysis from the LLMs' rationales. We further derive a rationale-free version of ARG by distillation, namely ARG-D, which services cost-sensitive scenarios without querying LLMs. Experiments on two real-world datasets demonstrate that ARG and ARG-D outperform three types of baseline methods, including SLM-based, LLM-based, and combinations of small and large language models.

1 Introduction

Fake news detection demands sensitivity to diverse clues and strong real-world knowledge, challenging current small language models. The paper investigates whether large language models can help and proposes using their rationales to advise small detectors.

Challenges: Fake news creators manipulate different parts of news using diverse strategies, so detection requires sensitivity to style, facts, commonsense, and real-world background.These requirements make both broad clue coverage and contextual understanding important.
Small language models: Small language models such as BERT and RoBERTa provide core news representations, but limited knowledge and capabilities constrain further improvement.BERT can struggle with news requiring knowledge absent from its pretraining corpus.
Large language models: Large language models are promising general task solvers, yet their potential for fake news detection remains underexplored.The paper asks whether LLMs can detect fake news using internal knowledge and capabilities and how to obtain better performance with them.
Proposed solution: ARG selectively injects insights from LLM-generated rationales into SLMs, while ARG-D distills this guidance for cost-sensitive use without LLM queries.Both methods are designed to combine the respective strengths of small and large language models.
Evaluation: ARG and ARG-D outperform existing SLM-only, LLM-only, and combined small–large language model methods on two real-world datasets.The paper also constructs a Chinese- and English-language rationale collection from GPT-3.5 for further research.
Role of LLMs: The paper finds that LLMs can analyze news from multiple perspectives but may fail to judge veracity correctly, motivating an advisor role rather than substitution.Figure 1 contrasts an incorrect LLM judgment with an SLM judgment improved by informative LLM rationales.

2 Is the LLM a Good Detector?

The study compares GPT-3.5-turbo with fine-tuned BERT under four prompting approaches and finds that LLMs analyze news richly but remain weaker detectors. Their difficulty integrating multi-perspective rationales motivates combining LLM analysis with SLM task-specific learning.

Comparison between Small and Large LMs: GPT-3.5-turbo underperforms fine-tuned BERT across all four prompting approaches on both datasets.BERT shows relative increases of 3.8%–11.3% in Chinese and 9.0%–34.6% in English over the LLM.
Comparison between Small and Large LMs: Few-shot prompting narrows the gap with BERT, but does not surpass it.Task-specific samples improve performance relative to zero-shot prompting.
Comparison between Small and Large LMs: CoT prompting generally improves performance, including a +17.3% gain for zero-shot English, but can also reduce performance.The mixed effects indicate that rationales require careful design and use.
Analysis on the Rationales from the LLM: LLM rationales analyze news from multiple perspectives, including textual description, commonsense, and factuality.The study manually categorizes 500 samples from each dataset and finds human-like analytical rationales across these perspectives.
Analysis on the Rationales from the LLM: Single-perspective prompting remains promising, performing comparably on Chinese and better on English for commonsense analysis.The comparison uses full-test-set coverage and excludes factuality to avoid hallucination impacts.
Analysis on the Rationales from the LLM: The LLM’s ineffective integration of diverse rationales limits its veracity judgment, motivating complementary use of LLM analytical capability and SLM task-specific knowledge.Voting experiments suggest adaptive combination could outperform LLM-only and SLM-only methods, positioning the LLM as an advisor rather than a substitute.

3 ARG: Adaptive Rationale Guidance Network for Fake News Detection

ARG bridges small and large language models by modeling interactions between news and LLM rationales, predicting LLM judgments, and adaptively weighting rationale contributions. ARG-D distills rationale-aware features into a rationale-free model for cost-sensitive prediction.

News-rationale interaction: ARG encodes news and textual-description and commonsense rationales, then learns news-rationale interactions to select useful analytical insights.The design uses separate BERT encoders and dual cross-attention, followed by average pooling of token-level outputs.
News-rationale interaction: The news-rationale interactor applies dual cross-attention so news and rationale representations exchange information.Average pooling converts cross-attention outputs into one-vector representations for subsequent processing.
LLM judgment prediction: ARG predicts the LLM’s veracity judgment from rationale representations to deepen the model’s understanding of rationale texts.For the textual-description branch, the rationale representation Rt is passed to an MLP-based LLM judgment predictor; the commonsense branch is analogous.
Rationale usefulness evaluation: ARG evaluates rationale usefulness from news-aware rationale vectors and reweights rationale-aware news features before final prediction.Usefulness labels are based on judgment correctness, while an MLP produces weights for reweighting the rationale-aware news vector.
Final prediction: ARG aggregates the attentive-pooled news vector with rationale-aware news vectors using learnable weights, then feeds the fusion vector to an MLP classifier.The total objective is a weighted sum of the stated loss terms.
Rationale-free distillation: ARG-D preserves the news encoder and attention module while distilling ARG’s aggregated fusion features into a rationale-aware feature simulator.This rationale-free version avoids querying the LLM at prediction time and targets cost-sensitive scenarios.

4 Evaluation

The evaluation compares LLM-only, SLM-only, and combined methods, finding that ARG performs best while distilled ARG-D retains gains without querying LLMs. Additional analyses examine module importance, rationale contributions, and cost-performance trade-offs.

Performance Comparison: ARG outperforms all compared methods in macro F1, while rationale-free ARG-D also surpasses them except ARG and its variants.The comparison includes LLM-only, SLM-only, and LLM+SLM baselines.
Ablation Study: Removing either the LLM Judgement Predictor or Rationale Usefulness Evaluator significantly decreases ARG performance.Even the weakest ARG variant outperforms other methods, highlighting the importance of news-rationale interaction.
Result Analysis: More than 77% of ARG-D’s overlapping correctly judged samples are shared with the LLM, indicating that ARG-D absorbs useful LLM judgment knowledge.Correct LLM judgments from both textual-description and commonsense perspectives contribute most to ARG-D’s additional gains.
Result Analysis: 20.4% and 22.1% of correct judgments are attributed to ARG-D itself.The authors speculate that the model produces new knowledge from incorrectly judged provided knowledge.
Cost Analysis in Practice: Querying ARG for only 23% of data raises performance to 0.784 in macro F1, matching the result of using ARG fully.The strategy uses ARG-D by default and sends selected samples, based on ARG-D confidence, to ARG.

5 Related Work

Related work covers social-context and content-based fake news detection, conventional reliance on pretrained representations, and emerging evidence that LLMs do not universally surpass well-trained small models.

Fake News Detection: Fake news detection methods are broadly categorized as social-context-based or content-based approaches.Social-context methods use propagation patterns and user feedback, whereas content-based methods analyze textual or visual clues.
Fake News Detection: Content-based methods commonly use pretrained models such as BERT and may supplement textual representations with knowledge bases or news environments.The paper combines large and small language models using textual content alone.
LLMs for Natural Language Understanding: Prior NLU results indicate that LLMs may lack comprehensive superiority over well-trained small models on some tasks.This paper contributes empirical evidence from fake news detection.

6 Conclusion and Discussion

The paper finds that GPT-3.5 can generate informative rationales but underperforms task-specific BERT, motivating ARG and rationale-free ARG-D to combine their complementary strengths. The authors identify current limitations and anticipate broader, more cost-friendly applications.

Conclusion: GPT-3.5 underperforms task-specific BERT but provides informative rationales that can complement small language models.The finding motivates using LLMs as advisors rather than substitutes for fine-tuned SLMs.
Conclusion: ARG flexibly combines small- and large-language-model strengths, while ARG-D removes the need to query LLMs in cost-sensitive scenarios.Experiments report superiority for both ARG and ARG-D.
Discussion: LLMs may struggle to use their internal capabilities effectively in applications requiring sophisticated real-world background.The discussion links this barrier to the need for improved prompting and deeper understanding of LLM mechanisms.
Discussion: The authors expect the solution to extend to other tasks and support more effective and cost-friendly use of LLMs.This is presented as a future expectation rather than an established result.
Limitations: The study does not examine other well-known LLMs, considers only perspectives summarized from LLM responses, and remains below oracle voting performance.These limitations leave open questions about model coverage, prompting perspectives, and achievable performance.

A Case Analysis

Case analyses show that ARG can resist misleading rationales, select a correct perspective when the LLM is only partially right, and sometimes succeed when both baseline and LLM predictions fail.

Table 6: ARG preserves the baseline’s correct prediction when the LLM provides a misleading rationale.This illustrates resistance to harmful LLM guidance.
Table 6: ARG selects the correct rationale when the baseline is wrong and the LLM is correct from only one of two perspectives.The cases demonstrate adaptive rationale selection.
Table 7: ARG identifies a correct prediction in a case where both the baseline and LLM fail.The authors associate this behavior with recognizing patterns linked to erroneous textual-description rationales.

B Prompting Examples

This section presents prompting templates and examples for several zero-shot and few-shot strategies, including perspective-specific and chain-of-thought prompting. It also describes using role-playing contexts to reduce GPT-3.5-turbo refusals on the English dataset.

Prompting templates: Tables 8 and 9 provide examples of zero-shot, zero-shot CoT, perspective-specific, few-shot, and few-shot CoT prompting.The examples include target news and GPT-3.5-turbo replies.
Few-shot prompting: Few-shot demonstrations show one real-news and one fake-news pair, while actual prompting uses multiple news-pair sets according to the required number of shots.
Prompting implementation: Role-playing contexts were added to English fake-news prompts because direct prompting had a notable likelihood of eliciting GPT-3.5-turbo refusals.The authors state that this technique decreased the refusal likelihood.
Prediction examples: Tables 6 and 7 compare testing cases involving the Baseline and an LLM using perspective-specific prompting, including cases with at least one correct prediction and cases where both predictions were incorrect.TD and CS denote textual description and commonsense perspectives.