PaperlayerView exact revision on arXiv

Source-linked AI summary

Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation

Kung-Hsiang Huang, Kathleen McKeown, Preslav Nakov, Yejin Choi, Heng Ji

arXiv:2203.05386v2cs.CL

TL;DR

Human-written disinformation remains difficult to detect with training data from neural fake-news generation, while human-authored examples can be difficult to collect and curate. The paper generates propaganda-informed training examples and reports that detectors trained on PROPANEWS perform best on two human-written datasets compared with other training datasets.

  • Problem

    Human-written disinformation can manipulate populations and has affected major events, motivating defenses informed by its styles and strategies.

    Source: §1 · p. 1

  • Method

    The paper generates detector-training articles informed by propaganda, including appeal to authority and loaded language.

    Source: §1 · p. 2, §2.2 · p. 4, §5.2 · p. 7

  • Results

    Both GROVER-LARGE and ROBERTA-LARGE perform best on SNOPES and POLITIFACT when trained on PROPANEWS, compared with other training datasets.

    Source: §5.2 · p. 6

  • Takeaways & Limitations

    PROPANEWS provides a released resource of 2.2K generated articles validated by humans for disinformation detection.

    Source: §1 · p. 2

    The method cannot generate all propaganda techniques, such as straw man, and is limited to English-only news articles.

    Source: §8 · p. 9

Abstract

from arXiv · show

Despite recent advances in detecting fake news generated by neural models, their results are not readily applicable to effective detection of human-written disinformation. What limits the successful transfer between them is the sizable gap between machine-generated fake news and human-authored ones, including the notable differences in terms of style and underlying intent. With this in mind, we propose a novel framework for generating training examples that are informed by the known styles and strategies of human-authored propaganda. Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles, while also incorporating propaganda techniques, such as appeal to authority and loaded language. In particular, we create a new training dataset, PropaNews, with 2,256 examples, which we release for future use. Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62 - 7.69% F1 score on two public datasets.

1 Introduction

Human-written disinformation is difficult to detect with training data generated by earlier methods, while collecting and verifying human-written examples is costly and limited in scale. The paper proposes propaganda-informed training data generation and reports improved detection of human-written disinformation compared with prior generation approaches.

1 Introduction

  • Existing collections of human-written fake news are limited in size and costly to curate, while suspicious-source articles require claim-level fact-checking and may be removed shortly after posting.The paper presents automatic generation as a complementary way to create training data.Source: §1 · p. 1
  • About 33% of the analyzed human-written disinformation articles used propaganda techniques, often involving emotion-triggering language or logical fallacies.The analysis covered around 40 articles spreading false claims.Source: §1 · p. 1
  • More than 55% of the analyzed articles mixed inaccurate information with correct information, making the few false claims more believable.The paper reports that all but one or two claims in these articles were factual.Source: §1 · p. 2
  • Earlier neural generation approaches produced articles with an overwhelmingly large proportion of false information and did not explicitly use propaganda.The paper identifies these as differences from human-authored disinformation.Source: §1 · p. 2
  • The proposed method replaces a salient sentence in an authentic article with plausible fake information, uses NLI-guided training and filtering, then adds propaganda techniques.The automated techniques include appeal to authority and loaded language.Source: §1 · p. 2
  • Detectors trained on the generated dataset were substantially better at spotting human-written disinformation than detectors trained on state-of-the-art generation approaches.The evaluation used two human-written fake-news datasets.Source: §1 · p. 2
  • PROPANEWS is a released dataset of 2.2K generated articles validated by humans.Crowdsourcing was used to validate generated texts and construct a gold-standard training dataset.Source: §1 · p. 2

2 Training Data Generation

Training-data generation has two stages: replace salient sentences with disinformation, then add propaganda techniques; intermediate pre-training adapts the models to news text.

2.1 Disinformation Generation

  • Salient sentences are identified by extractive-summarization scores, and the highest-scoring sentence is selected for replacement at inference.BART is trained to infill randomly masked sentences, then given an article with its highest-scoring sentence masked.Source: §2.1 · p. 3
  • Self-critical sequence training uses negative NLI entailment probability as reward to discourage generated text that preserves the masked sentence’s meaning.An NLI filter also removes outputs entailed by the masked sentence; incorporating the SCST loss reduced the invalid rate from 7.8% to 3.2%.Source: §2.1 · pp. 3–4

2.2 Propaganda Generation

  • Appeal-to-authority text is generated by prompting fine-tuned BART with an authority name and a template introducing that person’s statement.The best generated sequence is selected by lowest perplexity.Source: §2.2 · p. 4
  • Loaded-language generation uses 1,017 filtered examples from 2,547 annotated instances and a two-step BART process to insert and infill emotion-triggering language.The two-step approach reduced failures to reproduce unmasked context from around 25% to around 2%.Source: §2.2 · p. 5

2.3 Intermediate Pre-training

  • Intermediate pre-training on CNN/DM news articles adapts models because TIMELINE17 and the propaganda dataset are relatively small.CNN/DM is a large news summarization dataset.Source: §2.3 · p. 5

3 Our PROPANEWS Dataset

PROPANEWS is built from trustworthy, widely relevant news articles, with generated disinformation retained only after crowdsourced verification of inaccuracy.

3.1 Data Source

  • TIMELINE17 supplies 4,535 articles across 17 important news-event timelines, sourced from trustworthy outlets including The New York Times and The Guardian.The source was selected to keep unmanipulated article content genuine while covering events important to a general audience.Source: §3.1 · p. 5

3.2 Crowdsourcing for Data Curation

  • Around 400 Mechanical Turk workers completed approximately 2,000 tasks, and only generated sentences judged inaccurate after evidence review were retained in PROPANEWS.Accurate counterparts were discarded.Source: §3.2 · p. 5
  • Annotator agreement measured by WAWA reached 80.01% precision, 78.94% recall, and 79.47% F1, indicating moderate-to-high agreement.WAWA compares each annotator’s response with the majority-vote aggregate and micro-averages across samples.Source: §3.2 · pp. 5–6

4 Disinformation Detection

The detection experiments classify whether an article contains inaccurate information, comparing four neural and feature-based detectors under shared input and optimization settings.

4 Disinformation Detection

  • Four detectors are evaluated: HDSF, GROVER, BERT, and ROBERTA.HDSF uses discourse-level features, while GROVER, BERT, and ROBERTA use article representations with classification heads.Source: §4 · p. 6
  • HDSF predicts veracity from hierarchical discourse features such as dependency trees, while GROVER uses its discriminative model for detection.GROVER feeds CLS representations to a multilayer perceptron; BERT and ROBERTA classify from the first-token representation of the full article.Source: §4 · p. 6
  • All detectors use cross-entropy optimization, a maximum sequence length of 512, and LARGE model variants for comparison.The task is to determine whether an input article contains inaccurate information.Source: §4 · p. 6

5 Experiments

The experiments compare PROPANEWS with other generated and human-validated training sets on human-written disinformation detection, and find that detectors trained on PROPANEWS perform best. Ablations, distribution similarity, and plausibility ratings examine the contribution and quality of the generated data.

5.1 Data

  • PROPANEWS contains 2,256 articles with balanced real and fake portions; fake articles use appeal to authority (30%), loaded language (30%), or inaccurate information alone (40%).The data are split 1,256:500:500 for training, validation, and testing.Source: §5.1 · p. 6
  • The data comparisons include GROVER-GEN, FACTGEN, FAKEEVENT, and PN-SILVER; generators used the same authentic input articles for fair comparison.PN-SILVER resamples the generated data without annotator validation, while the -1SENT sets substitute one generated sentence for the salient sentence.Source: §5.1 · p. 6

5.2 Results and Discussion

  • Both GROVER-LARGE and ROBERTA-LARGE performed best on SNOPES and POLITIFACT when trained on PROPANEWS rather than other datasets.Table 3 reports AUC for the models and datasets; the supplied passages do not give the cell values.Source: §5.2 · p. 6, Table 3 · p. 7
  • PROPANEWS and GROVER-GEN received MAUVE scores of 17.1% and 13.7%, respectively, on POLITIFACT, suggesting PROPANEWS articles are closer to human-written disinformation.The authors connect this smaller gap to the detection advantage of their generated articles.Source: §5.2 · p. 7
  • Removing appeal to authority and loaded language reduced their benefit, while the no-propaganda PROPANEWS variant still outperformed other generated-data approaches.The authors interpret the latter comparison as evidence that their generated disinformation is closer to human-written examples.Source: §5.2 · p. 7
  • PROPANEWS plausibility averaged 2.25 out of 3, versus 2.15 for GROVER-GEN, a slight advantage in the worker ratings.Each set included 100 articles, with three workers rating each article.Source: §5.2 · p. 7
  • In POLITIFACT examples, ROBERTA-LARGE classified disinformative articles correctly after PN-SILVER training but incorrectly after GROVER-GEN training; both examples contained propaganda.The examples' propaganda was included in PN-SILVER but not GROVER-GEN.Source: §5.2 · p. 7, Table 4 · p. 8

6 Related Work

Prior neural fake-news generation largely targeted detection of machine-generated articles, while the paper focuses on human-written disinformation. It incorporates specific propaganda techniques into generated articles and positions its dataset for disinformation detection.

Fake News Generation and Detection

  • Prior neural generators aimed to defend against machine-generated fake news, but their abundant inaccurate content and lack of propaganda made them suboptimal for detecting human-written fake news (Zellers et al., 2019; Shu et al., 2021).The paper contrasts this with generating fake news that incorporates propaganda while preserving most correct information.Source: §6 · p. 8

Propaganda Generation and Detection

  • The authors describe their work as the first to incorporate specific propaganda techniques into generated articles, targeting human-written fake news rather than document- or fragment-level propaganda detection.They also note that PROPANEWS labels the exact offsets of disinformative passages, supporting future work on interpretable detection.Source: §6 · p. 8

7 Conclusions and Future Work

The paper presents PROPANEWS as generated disinformation closer to human-written fake news and reports improved detection on POLITIFACT and SNOPES. Future work would extend the approach across languages and propaganda techniques and explore other aspects of fake-news generation.

7 Conclusions and Future Work

  • The proposed method generates disinformation described as closer to human-written fake news.Source: §7 · p. 8
  • Evaluation on the human-written POLITIFACT and SNOPES datasets demonstrates better detection performance using PROPANEWS.The conclusion presents PROPANEWS as a resource for human-written fake-news detection.Source: §7 · p. 8
  • Future work will extend the approach to other languages and propaganda techniques and examine novelty, elaboration, and engaging linguistic style.Source: §7 · p. 9

8 Limitations

Increasing PN-SILVER initially improves detection on POLITIFACT, but performance plateaus at five times the dataset size, and the generated articles remain distinct from human-crafted fake news. The method also cannot generate all propaganda techniques and is limited to English news.

8 Limitations

  • The remaining gap may reflect differences in style, intent, and falsehood, since the generated content is 100% false.The authors describe these as likely sources of the gap.Source: §8 · p. 9
  • The generator cannot produce some propaganda techniques, such as straw man, and is limited to English-language news.The authors state it is not generic enough to handle all propaganda techniques in a unified framework.Source: §8 · p. 9
  • Performance improves initially but plateaus after PN-SILVER reaches five times its size, indicating a remaining gap from human-crafted fake news.The comparison uses ROBERTA-LARGE on POLITIFACT, with SNOPES as the human-written training data reference.Source: Figure 3 · p. 9, §8 · p. 9

9 Ethical Statement and Broader Impact

The paper discusses the risks of generating propaganda-informed disinformation and reports analyses of propaganda, detector behavior, and remaining detection challenges. It also describes how its authority-appeal generation is diversified.

9 Ethical Statement and Broader Impact

  • The generator may produce toxic text and could be misused to create harmful disinformation; the authors released detectors and generated data but not the generator.They identify possible political, social, medical, financial, and legal harms.Source: §9 · p. 9

A Distribution of Propaganda

B Additional Research Questions

  • Detectors trained without appeal to authority and loaded language still identify human-written false articles, while adding those techniques provides additional evidence.The comparison is reported for PROPANEWS versus PROPANEWS w/o AA & LL.Source: B Additional Research Questions · p. 13
  • Real PolitiFact articles have 7.3% MAUVE similarity to generated text, and the metric suggests they contain little loaded language or appeal to authority.The paper notes that some real articles may contain propaganda.Source: B Additional Research Questions · p. 13

C.1 Remaining Challenges

  • Misclassifications reflect static-knowledge gaps (~30%), dynamic-knowledge gaps (~48%), and missing multi-document reasoning.The analysis attributes the remaining incorrect detections to lack of multi-document reasoning.Source: §C.1 · pp. 13–14

D Qualitative Examples of Generated Articles

E Appeal to Authority Details

  • For appeal to authority, the method selects the lowest-perplexity sequence from Wikidata authorities, then diversifies templates to avoid repeated “confirms that” cues.Diversification includes reordering sentence elements and varying verbs and prepositions.Source: E Appeal to Authority Details · p. 14

F Intermediate Pre-training Details

The appendix details intermediate pre-training, detector benchmarking, and human validation. It describes differing pre-training objectives, qualitative detector comparisons, and the validation criteria applied to generated text.

F Intermediate Pre-training Details

  • Intermediate pre-training uses CNN/DM news articles; disinformation generation omits Ls to focus on coherent sentences, while loaded-language training teaches insertion locations.Loaded-language samples are gathered using dependency parses of adjective–noun and adverb–verb relationships.Source: F Intermediate Pre-training Details · p. 14

G Benchmarking Detectors

  • ROBERTA and GROVER outperform BERT on PROPANEWS, while HDSF performs worse than the other three detectors.The authors suggest news-domain pre-training benefits detection more than explicit discourse-level feature modeling.Source: G Benchmarking Detectors · p. 14, Table 5 · p. 14

H Human Validation Details

  • AMT validation used screened workers from the United States and Canada; generated text was labeled inaccurate if it contained any false information.Workers needed an acceptance rate above 95% and more than 100 prior accepted HITs.Source: H Human Validation Details · pp. 14–15

I Statistics of the Evaluation Datasets

The evaluation-dataset statistics exclude articles whose URLs were no longer accessible, so the reported counts differ from the original papers. The appendix also lists detector training configurations and hardware.

I Statistics of the Evaluation Datasets

J Detector Implementation Details

  • BERT and ROBERTA use AdamW with batch size 2 and gradient accumulation of 8; GROVER and HDSF follow their original training settings.Pre-trained parameters use learning rate 5e-5 and weight decay 1e-5; other parameters use 1e-3 for both.Source: J Detector Implementation Details · pp. 15–16
  • Experiments ran on an NVIDIA Tesla V100; BERT and ROBERTA training took about 2–3 hours, compared with about 1 hour for GROVER.The implementation used PyTorch and Transformers, except GROVER, which used TensorFlow.Source: J Detector Implementation Details · p. 16

K Human Evaluation Details

The appendix describes an AMT-based human evaluation of article quality and reports higher quality for the proposed approach. It also details PROPANEWS artifact licensing and release plans.

K Human Evaluation Details

  • AMT workers rated article plausibility and, for PROPANEWS only, how much a propaganda sentence influenced that judgment.Low, medium, and high responses correspond to scores 1, 2, and 3.Source: K Human Evaluation Details · p. 16
  • The authors report that their approach produces higher-quality fake news; Figure 5 breaks down Q1 vote counts by method.The figure’s x-axis shows evaluator vote counts and its y-axis lists methods.Source: K Human Evaluation Details · p. 16, Figure 5 · p. 16

L Scientific Artifacts

  • PROPANEWS reuses Timeline17, and the authors plan compatible-license release of the dataset and detectors alongside open-source-licensed tools and models.Source: L Scientific Artifacts · p. 16
  • Table 8 notes that other approaches generate much inaccurate information, while FACTGEN appears low-quality, possibly because checkpoints were unavailable and training details may have been missing.Source: Table 8 · p. 17

Related papers

Loading 2203.05386v2…