PaperlayerView exact revision on arXiv

Source-linked AI summary

Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks

Jiaying Wu, Jiafeng Guo, Bryan Hooi

arXiv:2310.10830v2cs.CL

TL;DR

LLM-powered style attacks can make fake news resemble trustworthy sources, exposing a robustness gap in text-based detectors. The paper introduces SheepDog, which combines style-diverse reframings, style-agnostic training, and content-focused attributions; experiments across three benchmarks demonstrate style robustness and effectiveness across settings and backbones.

  • Problem

    LLM-empowered style attacks can mimic reputable news styles, while text-based detectors show limited robustness to stylistic variations.

  • Method

    SheepDog combines LLM-empowered news reframings, style-agnostic training, and content-focused veracity attribution prediction in a multi-task detector.

  • Results

    SheepDog demonstrates robustness and effectiveness across style-based adversarial settings, news reframing prompts, three real-world benchmarks, and representative backbones.

  • Takeaways & Limitations

    Fake news detection should emphasize content veracity over stylistic cues when article styles can be rapidly manipulated by LLMs.

  • Takeaways & Limitations

    Performance gains on LUN may partly stem from dataset-specific stylistic features.

Abstract

from arXiv · show

It is commonly perceived that fake news and real news exhibit distinct writing styles, such as the use of sensationalist versus objective language. However, we emphasize that style-related features can also be exploited for style-based attacks. Notably, the advent of powerful Large Language Models (LLMs) has empowered malicious actors to mimic the style of trustworthy news sources, doing so swiftly, cost-effectively, and at scale. Our analysis reveals that LLM-camouflaged fake news content significantly undermines the effectiveness of state-of-the-art text-based detectors (up to 38% decrease in F1 Score), implying a severe vulnerability to stylistic variations. To address this, we introduce SheepDog, a style-robust fake news detector that prioritizes content over style in determining news veracity. SheepDog achieves this resilience through (1) LLM-empowered news reframings that inject style diversity into the training process by customizing articles to match different styles; (2) a style-agnostic training scheme that ensures consistent veracity predictions across style-diverse reframings; and (3) content-focused veracity attributions that distill content-centric guidelines from LLMs for debunking fake news, offering supplementary cues and potential intepretability that assist veracity prediction. Extensive experiments on three real-world benchmarks demonstrate SheepDog's style robustness and adaptability to various backbones.

1 INTRODUCTION

Style differences can help detect fake news but also expose detectors to LLM-enabled attacks that mimic reputable publishers. SheepDog addresses this vulnerability by emphasizing content through style-diverse reframings, style-agnostic training, and content-focused veracity attributions.

  • LLMs enable malicious actors to disguise fake news in the styles of reputable news sources, allowing style-based attacks against automated detectors.
  • 38% decline in F1 Score is observed for some state-of-the-art text-based detectors under adversarial style-based attacks.
  • SheepDog uses LLM-empowered reframings to create style-diverse training expressions while preserving article content.
  • Style-agnostic training encourages consistent veracity predictions across each article and its style-diverse reframings.
  • Content-focused veracity attributions convert LLM-generated, content-centric debunking rationales into pseudo-labels that provide additional weak supervision.
  • Extensive experiments report superior style robustness and consistent performance gains across representative language-model backbones.

2 RELATED WORK

Fake news detection research has used neural architectures and text features alongside auxiliary signals such as comments, knowledge bases, temporal patterns, and social graphs. Recent work also addresses temporal, entity, and domain shifts.

  • Fake news detectors use lexical and sentiment features within articles, alongside auxiliary information from users, knowledge bases, temporal patterns, and social graphs.
  • Recent studies address robustness challenges including temporal shift, entity bias, and domain shift in fake news detection.

3 PROBLEM DEFINITION

The paper formulates text-based fake news detection as predicting binary veracity labels for unlabeled news pieces in a dataset with labeled training articles. The setting focuses on textual information and style-related issues.

  • Veracity labels are binary, representing either real news or fake news.
  • The problem is restricted to a text-based setting because the paper focuses on style-related issues.
  • The task is to predict veracity labels for unlabeled news pieces using a dataset with labeled training articles.

4 LLM-EMPOWERED STYLE ATTACKS

The paper evaluates LLM-empowered style attacks by restyling real and fake news toward contrasting publisher styles, then measuring detector robustness. Existing detectors show substantial degradation, motivating style-agnostic detection focused on content.

  • LLM restyling prompts allow malicious users to disguise fake news in styles associated with reliable publishers.
  • Real test news is rewritten in tabloid style, while fake news is rewritten in mainstream-source style to simulate adversarial publisher-style changes.
  • The evaluation covers 13 representative text-based detectors, including task-specific architectures, fine-tuned language models, and zero-shot LLMs.
  • 38.3% F1 Score decline is observed on the adversarial test set, demonstrating substantial vulnerability among state-of-the-art text-based detectors.
  • LLMs show inferior fake news detection performance compared with text-based detectors and pretrained LMs fine-tuned for this task.
  • These findings motivate prioritizing news content over style because stylistically influenced detectors struggle when article styles evolve or are manipulated.

5 PROPOSED APPROACH

SheepDog combines style-agnostic training with content-focused veracity attribution prediction to assess news based on content rather than writing style. It uses LLM-generated reframings and auxiliary attribution supervision within an end-to-end, modular framework.

  • 5 PROPOSED APPROACH: SheepDog combines style-agnostic training and content-focused veracity attribution prediction in a multi-task framework.The method is designed to integrate with LM and LLM backbones.
  • 5.2 Style-Agnostic Training: LLMs reframe each training article into reliable- and unreliable-style expressions while preserving its underlying content.Prompts use objective, professional, neutral, emotionally triggering, and sensational styles.
  • 5.2 Style-Agnostic Training: The style alignment objective encourages identical veracity predictions for an article and its reliable- and unreliable-style reframings.The LM produces representations and an MLP produces two-logit real/fake predictions for each version; KL divergence aligns reframed predictions with the original.
  • 5.3 Content-Focused Veracity Attributions: Content-focused LLM attributions provide pseudo-labels for fake-news rationales and supplementary supervision for veracity prediction.Real news and its reframings receive all-zero attribution labels because the rationale set contains fake-news indicators.
  • 5.3 Content-Focused Veracity Attributions: The framework adds content-centric guidance and potential explanations for articles identified as fake during inference.The approach is trained end-to-end with the veracity predictor and attribution predictor optimized simultaneously.
  • 5.4 Final Objective Function of SheepDog: The final objective linearly combines style alignment, news classification, and veracity attribution losses.Attribution scores use binary cross entropy, while the news-classification loss uses cross entropy.

6 EXPERIMENTS

The experiments evaluate SheepDog across real-world benchmarks, adversarial style attacks, unperturbed articles, model backbones, and controlled implementations. Results emphasize robustness against LLM-empowered style attacks and comparative performance across detector families.

  • Experimental Setup: Experiments use FakeNewsNet’s PolitiFact and GossipCop datasets and the Labeled Unreliable News dataset.The evaluation covers political discourse, celebrity gossip, and unreliable-news categories including satire, hoax, and propaganda.
  • Experimental Setup: SheepDog is compared with thirteen representative baselines spanning text-based detectors, fine-tuned language models, and zero-shot large language models.The baselines include neural task-specific architectures, pretrained language models, and GPT-3.5, InstructGPT, and LLaMA2-13B.
  • Evaluation Protocol: Performance is reported using Accuracy (%) and macro-F1 Score (%), with non-LLM results averaged over 10 runs.LLM zero-shot predictions use greedy decoding and are evaluated once.
  • Style-Attack Evaluation: Four adversarial test sets evaluate detector performance when trustworthy and unreliable news are reframed into contrasting publisher styles.The attacks use tabloid styles for real news and mainstream-source styles for fake news, producing sets A through D.
  • Style-Attack Evaluation: All baseline methods are highly susceptible to LLM-empowered style attacks, while SheepDog significantly outperforms the most competitive baseline across the three benchmarks.The reported average improvements are 2.59%, 2.77%, and 15.70% across the three benchmarks.

6.3 Effectiveness on Unperturbed Articles

On unperturbed articles, SheepDog matches or surpasses the strongest baseline while retaining robustness across backbones and reframing prompts. Ablations indicate that diverse reframings and content-focused attributions contribute distinct benefits.

  • Unperturbed Articles: SheepDog matches the strongest baseline on PolitiFact and GossipCop and surpasses it on LUN for both accuracy and F1 score.This result concerns the original, unaltered test articles.
  • Backbone Adaptability: SheepDog substantially enhances each evaluated LM backbone and remains more style-robust with both closed-source and open-source LLM backbones.The evaluated LM backbones are RoBERTa, BERT, and DeBERTa; the LLM backbones are GPT-3.5 and LLaMA2-13B.
  • Ablation Study: Removing LLM-generated news reframings yields only slight improvements over fine-tuned RoBERTa, indicating their key role in SheepDog’s robustness.This comparison is represented by SheepDog-R in the ablation study.
  • Ablation Study: Content-focused veracity attributions guide the model toward prioritizing content over style and provide explanatory outputs that facilitate human verification.The attribution predictor contributes supplementary inference-time explanations.
  • Prompt Stability: SheepDog maintains stable and significant improvements across different reframing-prompt combinations.The approach samples between two reliable-style and two unreliable-style reframings.

6.7 Case Study

A case study contrasts a baseline’s behavior with SheepDog’s on a fake article and its style-transformed versions. The example connects style reliance with publisher and news-type distribution shifts in LUN.

  • Case Study: The case study examines a LUN article falsely claiming that cannabis oil treats cancer despite contradicting established medical knowledge.The article is presented as an example of deceptive content requiring veracity assessment.
  • Case Study: RoBERTa correctly flags the original article but misclassifies two style-transformed adversarial versions as real news.The baseline therefore changes its prediction when the article’s style is altered.
  • Case Study: SheepDog maintains consistent veracity prediction and top-predicted debunking attribution across the original article and LLM-camouflaged counterparts.The figure is used to illustrate both prediction consistency and attribution consistency.
  • Interpretation: LUN’s larger gains are associated with publisher and news-type styles, including a publisher-disjoint train-test distribution shift.Baseline style features fail to generalize under these shifts, whereas SheepDog is described as unaffected by publisher-style changes.

7 CONCLUSION AND FUTURE WORK

SheepDog combines style-agnostic training and content-focused veracity attribution prediction to improve robustness against style-based adversarial settings across benchmarks and backbones.

  • SheepDog integrates style-agnostic training and content-focused veracity attribution prediction within a multi-task learning paradigm.
  • Experiments on three real-world benchmarks demonstrate robustness across style-based adversarial settings, news reframing prompts, and representative backbones.
  • The approach provides a foundation for more resilient and adaptable models and may extend to multi-modal scenarios.

A DISCUSSION: EFFECT OF REFRAMINGS ON LLM STYLE ROBUSTNESS

The experiments show that style-diverse reframings improve robustness in both in-context learning and fine-tuning, although long articles limit the benefits of LLM in-context learning and LLaMA2 fine-tuning remains comparatively weaker.

  • In-Context Learning: Style-diverse reframings consistently improve GPT-3.5 in-context style robustness over the corresponding unreframed setup.GPT-3.5+ICL-2-R outperforms GPT-3.5+ICL-2.
  • In-Context Learning: LLM in-context learning does not provide clear benefits over zero-shot fake news detection because long articles constrain demonstrations and produce overly lengthy prompts.
  • Fine-Tuning: Both LLaMA2 variants use QLoRA fine-tuning for 1 epoch with batch size 16 and learning rate 1 × 10^-4.
  • Fine-Tuning: Fine-tuning improves LLaMA2-13B task-specific fake news detection, while adding style-diverse reframings further improves style robustness.
  • Fine-Tuning: LLaMA2-13B+FT-R remains below GPT-3.5 on the LUN adversarial test set and below task-specific detectors and fully fine-tuned language models.

B LLM PROMPTING CONFIGURATIONS

The appendix specifies baseline models, prompting formats, reframing templates, and LLM-generated content-focused veracity attributions used in SheepDog’s experiments.

  • B.1 LLM Baselines: The LLM baselines are GPT-3.5, InstructGPT, and LLaMA2-13B.
  • B.1 LLM Baselines: GPT-3.5 and InstructGPT use APIs with temperature 0, while LLaMA2-13B uses HuggingFace weights and greedy decoding.
  • B.1 LLM Baselines: The zero-shot prompt asks whether an article contains real or fake news and requests a one-word label followed by an explanation.
  • B.2 Obtaining SheepDog’s News Reframings: GPT-3.5 generates reliable- and unreliable-style reframings using four templates, with objective and neutral tones for reliable styles and emotionally triggering and sensational tones for unreliable styles.
  • B.2 Obtaining SheepDog’s News Reframings: Reframing generation uses temperature 0.7 and a maximum response length of 512 tokens.
  • B.3 Obtaining SheepDog’s Veracity Attributions: An LLM produces explanatory outputs for each fake news training article using predefined content-focused rationales for debunking.
  • B.3 Obtaining SheepDog’s Veracity Attributions: The rationales cover source credibility, consistency with reputable sources, false or misleading information, and biased opinion.
  • B.3 Obtaining SheepDog’s Veracity Attributions: The attribution prompt asks the LLM to select applicable problems from the four rationales, order multiple problems by relevance, or answer “No problems.”

C ANALYSIS ON CONTENT CONSISTENCY OF LLM-EMPOWERED NEWS REFRAMINGS

The paper evaluates whether LLM-generated reframings preserve central factual content while introducing stylistic variation, finding high claim consistency across two benchmark datasets.

  • LLM-generated reframings introduce different tones while preserving the news content of the original articles.
  • Claim entailment measures whether the original and reframed articles support each other’s central factual claims.
  • GPT-3.5 extracts central factual claims and predicts entailment between articles because news articles are longer and logically more complex than standard NLI sentence pairs.
  • The claim-extraction example summarizes concerns about China’s undervalued yuan and calls for policy action to prevent currency and trade wars.
  • The evaluation covers GossipCop and LUN to estimate content consistency at larger scale.
  • 86.20% to 89.22% of claim entailment is observed across both datasets, indicating reasonably high consistency between original articles and objective-style reframings.
  • The appendix includes an example of GPT-3.5-generated reliable-style reframing and an example of content-focused veracity attributions.
Loading 2310.10830v2…