PaperlayerView exact revision on arXiv

Source-linked AI summary

Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlines

Saadia Gabriel, Skyler Hallinan, Maarten Sap, Pemi Nguyen, Franziska Roesner, Eunsol Choi, Yejin Choi

arXiv:2104.08790v4cs.CL

TL;DR

Misinformation research needs to account for readers’ interpretations and reactions, not only whether headlines are true. This paper introduces Misinfo Reaction Frames and a crowdsourced corpus, and reports that generated frames can affect readers’ trust in headlines. It also identifies limits to modeling reactions from headlines alone.

  • Problem

    Understanding how headlines influence readers requires modeling perceived intent and reactions beyond fact-checking, since readers may share misinformation even when they suspect it is false.

    Source: §1 · pp. 1–2

  • Method

    The authors define Misinfo Reaction Frames, collect 69.8k annotated implications for 25.1k Covid-19, climate, and cancer headlines, and train models to predict frame dimensions.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Machine-generated MRF inferences affected readers’ trust in headlines, and the best model’s trustworthiness-label correlation was Pearson’s r=0.24, p=0.018.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Generated reaction frames may support downstream work on event causality, empathetic response generation, and counter-narratives.

    Source: §7 · p. 9

    Because the study focuses on headlines, its context is limited and headline intent may differ from the corresponding article’s intent.

    Source: §6 · p. 9

Abstract

from arXiv · show

Even to a simple and short news headline, readers react in a multitude of ways: cognitively (e.g. inferring the writer's intent), emotionally (e.g. feeling distrust), and behaviorally (e.g. sharing the news with their friends). Such reactions are instantaneous and yet complex, as they rely on factors that go beyond interpreting factual content of news. We propose Misinfo Reaction Frames (MRF), a pragmatic formalism for modeling how readers might react to a news headline. In contrast to categorical schema, our free-text dimensions provide a more nuanced way of understanding intent beyond being benign or malicious. We also introduce a Misinfo Reaction Frames corpus, a crowdsourced dataset of reactions to over 25k news headlines focusing on global crises: the Covid-19 pandemic, climate change, and cancer. Empirical results confirm that it is indeed possible for neural models to predict the prominent patterns of readers' reactions to previously unseen news headlines. Additionally, our user study shows that displaying machine-generated MRF implications alongside news headlines to readers can increase their trust in real news while decreasing their trust in misinformation. Our work demonstrates the feasibility and importance of pragmatic inferences on news headlines to help enhance AI-guided misinformation detection and mitigation.

1 Introduction

The paper argues that understanding misinformation requires modeling readers’ reactions and perceived intent, not only headline veracity. It introduces Misinfo Reaction Frames and a corpus, then evaluates reaction prediction and reader-trust effects.

1 Introduction

  • Preventing misinformation spread requires more than determining veracity, because readers may share a pandemic conspiracy headline even when they suspect it is misinformation.The paper notes that misinformation can also reinforce sociopolitical divisions, worsen public-health risks, and undermine public education about global crises.Source: §1 · pp. 1–2
  • MRF is a pragmatic formalism and corpus for modeling reader reactions and perceived author intent, with 69.8k implications for 25.1k Covid-19, climate, and cancer headlines.Source: §1 · p. 2
  • 85% F1: reaction inference models correctly label headline veracity and can infer commonsense implications, though nuanced implications remain difficult.The paper gives the example of inferring that a cat’s arrest for violating curfew means lockdowns are enforced, but notes difficulty with an implication of government incompetence.Source: §1 · p. 2
  • 86% F1: finetuning the MRF model on 574 annotated cancer examples tests generalization to a new domain.Source: §1 · p. 2
  • In a user study, machine-generated MRF inferences affected readers’ trust in headlines; the best model’s trustworthiness-label correlation was Pearson’s r=0.24, p=0.018.Source: §1 · p. 2
  • The authors publicly release the MRF corpus and trained models to support further work.Source: §1 · p. 2

2 Misinfo Reaction Frames

MRF takes a reader-centric approach to interpreting headline intent because writers’ intent can be difficult to recover in social-media news sharing. It represents pragmatic implications with free-text reactions grounded in common themes.

Motivation for Our Formalism

  • MRF models a headline’s potential impact on readers, complementing misinformation approaches focused on linguistic or social-media-derived features.Source: §2 · p. 3
  • Rather than model both a pragmatic speaker and listener, MRF interprets headline intent from the reader’s perspective because writer intent is challenging to recover in dynamic social-media sharing.Source: §2 · p. 3

Defining the Frame Structure

  • Annotations use free-text implications tied to one of seven common themes, such as technology or government entities, to ground reactions in real-world concerns.Source: §2 · pp. 3–4

3 Misinfo Reaction Frames Corpus

The MRF corpus contains crowdsourced reactions to headlines about Covid-19, climate change, and cancer. Its collection combines news data from these domains with structured annotation and quality-control procedures.

3 Misinfo Reaction Frames Corpus

  • The corpus contains 69,885 implications from 25,164 headlines: 11,757 Covid-19, 12,733 climate, and 674 cancer headlines.Source: §3 · p. 4

3.1 News Data Collection

  • The collection covers reliable and unreliable headlines on Covid-19 and climate change, plus cancer headlines used to test out-of-domain performance.The task focuses on misinformation that may spread unintentionally, distinguishing it from disinformation, which assumes malicious intent.Source: §3.1 · p. 4

3.2 Annotation Process

  • Crowdsourced annotators rated headline veracity and spread likelihood and supplied free-text responses; quality control retained 80 workers and yielded 79% pairwise agreement and Cohen’s κ=.51.Source: §3.2 · p. 5

4 Modeling Reaction Frames

The paper models reaction-frame dimensions from headlines using generative and discriminative language models. It also uses masked fine-tuning to improve cross-domain generalization.

4 Modeling Reaction Frames

  • The modeling setup uses generative language models for free-text inferences and generative or BERT-based discriminative models for categorical inferences, compared with a nearest-neighbor retrieval baseline.The baseline retrieves gold implications aligned with the most similar training headline.Source: §4 · p. 5

4.1 Controlled Generation

  • Generative models receive a headline with topic and reaction-dimension control tokens and produce a short inference for that dimension.Inference proceeds from the topic token until the end token; unknown-topic data can jointly predict the topic and inference.Source: §4.1 · p. 5

4.2 Classification

  • Discriminative models take the headline as input and output a categorical inference.The input is framed with model-specific [CLS] and [SEP] tokens.Source: §4.2 · p. 6

4.3 Training

  • Training uses cross-entropy loss, with GPT-2 loss computed over the full sequence and T5 loss computed only over the output.The probability in the loss is given by the language model.Source: §4.3 · p. 6

4.4 Masked Fine-Tuning

  • Masked fine-tuning targets domain-specific keyphrases by masking the top 100 phrases for each domain in Covid-19 and climate training examples before continued training.The phrases are selected using TextRank and domain-specificity based on the complement of the domains’ shared keyphrases.Source: §4.4 · p. 6

5 Experiments

Experiments evaluate whether MRF can predict likely reactions, counter misinformation, and detect it, using automatic metrics and human evaluation.

5.2.1 Automatic Metrics

  • Generation and classification were evaluated with BLEU-4, BERTScore, and macro-averaged precision, recall, and F1, respectively.BLEU-4 measures n-gram overlap, while BERTScore measures semantic similarity to references.Source: §5.2.1 · p. 6

5.2.2 Human Evaluation

  • Human evaluation sampled unique T5 and GPT-2 large implications for 196 headlines, with three judgments per headline and before-and-after trust ratings.Annotators also rated implication quality and perceived sociopolitical acceptability.Source: §5.2.2 · pp. 6–7

5.1 Setup

  • The test split used 2021 and the final two months of 2020, separating train and test by date to reduce topical and news-event overlap.The splits had no exact overlap.Source: §5.2.2 · p. 6, §5.1 · p. 6

5.3 Generating Reaction Frames

  • T5-large generations received slightly higher quality ratings, but up to 25.34% of judgments rated generations socially unacceptable.Most generations were rated socially acceptable.Source: §5.3 · p. 7
  • T5-base trust shifts correlated significantly with gold labels (Pearson’s r = .24), while annotators reported that generated writer intents made real news seem more trustworthy and misinformation less trustworthy.Across most other models, trust decreased for both real news and misinformation.Source: §5.4 · p. 7, §5.3 · p. 7

5.4 Detecting Misinformation

  • Prop-BERT zero-shot classification had considerably lower F1 than task-specific models, likely because real and misinformation news both use propaganda techniques.The zero-shot rule labels news as real when no propaganda technique is detected, and misinformation otherwise.Source: §5.4 · p. 8
  • 85.26 max F1 compared with human F1 of 75.21; Covid-BERT improved 5.46% over BERT without domain-specific pretraining.The authors report that misinformation detection remained nontrivial for large-scale models.Source: §5.4 · p. 8
  • On 100 cancer-related real and misinformation health news headlines, one epoch of masked fine-tuning substantially boosted unsupervised generative MRF models, with up to 43.49% performance increase and F1 similar to full-data domains.Source: §5.4 · p. 8

6 Future Directions and Limitations of Reaction Frames

MRF offers ways to study perceived intent and impact of misinformation, including potential uses in counter-narratives and spread analysis. The authors also identify limits tied to headline context, annotator bias, and the study’s language and worker population.

6 Future Directions and Limitations of Reaction Frames

  • Annotated sharing and reading likelihood can estimate propagation, while writer-intent labels can reveal themes across domains and unseen headlines.The authors give mistrust of vaccination across medical domains as an example of a common perceived intent.Source: §6 · p. 9
  • Perceived labels can identify misinformation that may fool knowledgeable readers and inform countering systems or adversarial examples; future work could categorize severity by potential harm.The passage contrasts false headlines that incite violence or discourage vaccination with false headlines having more benign consequences, such as satire.Source: §6 · p. 9
  • MRF explanations can provide effective counter-narratives by discouraging trust in misinformation.The authors encourage further model improvements, including through domain knowledge.Source: §6 · p. 9
  • Headline-focused frames have limited context because a headline’s intent may differ from the corresponding article’s intent.The authors note that readers often share headlines without clicking, while suggesting extension to full articles as future work.Source: §6 · p. 9
  • Annotators may bring political or social biases, and such biases can also appear in model pretraining data.The authors recommend diverse annotator pools and suggest that future modeling could mitigate harmful bias depending on use case.Source: §6 · p. 9
  • The study uses English-language news and US-based workers, so interpretations may differ across languages and cultures.Source: §6 · p. 9

7 Conclusion

The paper introduces MRF as a pragmatic formalism for understanding readers’ perceptions of news reliability. It reports that machine-generated frames can change reader perceptions, while noting remaining modeling work and potential downstream uses.

7 Conclusion

  • MRF is a pragmatic formalism for understanding reader perception of news reliability.The authors describe possible downstream uses including event-causality understanding, empathetic response generation, and counter-narratives.Source: §7 · p. 9
  • Machine-generated reaction frames can change readers’ perceptions, and large-scale language models can discern real news from misinformation, though further work remains.Source: §7 · p. 9
  • Biases can manifest even in informed readers’ viewpoints, and removing examples or controlling for annotator neutrality does not erase their causes.Source: §7 · p. 9

8 Ethical Considerations

The authors discuss both the potential misuse of frame-based interpretations and the value of studying reader reactions to misinformation. They urge caution about treating annotations as universal truths and identify possible broader applications for mitigating spread.

8 Ethical Considerations

  • Frame-based machine-generated reader interpretations could be misused to produce more persuasive misinformation.The authors do not consider this an imminent threat, but urge systemic safety and fairness measures before large-scale deployment.Source: §8 · p. 10
  • Annotations may reflect annotators’ perceptions and beliefs rather than universal truths, so generalizing across social and cultural contexts warrants caution.The authors note demographic homogeneity among online crowdsourcing workers.Source: §8 · p. 10
  • MRF was designed to identify and predict high-impact misinformation that is more likely to spread, supporting future research on danger factors and mitigation systems.Source: §8 · p. 10

A.1 Additional Annotation Details

The annotation process retains qualified workers’ annotations, elicits free-text implications alongside categorical judgments, and uses English-language headlines, including translated international headlines.

A.1 Additional Annotation Details

  • The dataset retains annotations from qualified workers and discards annotations from disqualified workers; headlines with no annotations due to truncation or vagueness are removed.Workers were paid at specified per-hit rates and consented by accepting tasks after reading their descriptions.Source: §A.1 · p. 15
  • For writer intent, annotators assessed seven predefined themes and supplied one to three implications for relevant themes; reader perception and action elicited one or two implications each.Source: §A.1 · p. 15
  • All headlines are in English, including international news headlines translated from multiple original languages in the Poynter database.Source: §A.1 · p. 15

A.1.1 Full Instructions to Annotators

Annotators were asked to infer what a news event might imply, including readers’ reactions and the fragment author’s intent. They were instructed to consider broad references and multiple possible topics.

A.1.1 Full Instructions to Annotators

  • Annotators were asked to base answers on the most generic form of any organization or location named in the event fragment.For example, they should write “government” or “health agency” rather than “CDC.”Source: §A.1.1 · p. 15
  • The task asked annotators to consider both the reaction a news event might invoke and the fragment author’s intent.The instructions note that the article may contain misinformation.Source: §A.1.1 · p. 15
  • Annotators were encouraged to consider multiple topic categories when describing reactions and intent, rather than limiting implications to one topic.The instructions give technology and society as an example of topics that may overlap.Source: §A.1.1 · p. 15

A.2 Post-processing of Annotations

Annotations were filtered, deduplicated, and sometimes reclassified before missing values were handled. Headlines without free-text annotations were removed, while some missing reader annotations were inferred from other MRF variables.

A.2 Post-processing of Annotations

  • Annotations on the same MRF dimension were deduplicated when ROUGE-2 overlap was at least .8, and writer-intent annotations were removed when they overlapped the headline by at least .8.Keyword filtering also removed climate-related headlines mentioning toxic work environments or political climates; the authors note that some corner cases may remain.Source: §A.2 · p. 15
  • Some perception and action labels were reclassified based on annotation content, while single-word annotations with common misspellings were removed.Emotion-word action labels could become perceptions, and perception labels containing “want” could become actions.Source: §A.2 · p. 15
  • Writer-intent annotations had to contain at least three words, while reader perception and action annotations had to contain at least three characters.Automatic spellchecking was avoided because it could alter names, such as changing “biden” to “widen.”Source: §A.2 · p. 15

A.3 Experimental Setup and Model Hyperparameters

The appendix reports model settings and analyses of sharing, reading, and trust effects, then relates MRF to prior work on rhetorical framing and misinformation. It also discusses how generated writer-intent implications can affect perceived trustworthiness, with errors as an important caveat.

A.3 Experimental Setup and Model Hyperparameters

  • The experimental setup used a single GPU per run, with training times ranging from about 30 minutes per BERT epoch to four hours per GPT-2 large epoch.Generative-model inference took about 10–20 minutes, and hyperparameters were manually tuned.Source: §A.3 · p. 16

A.3.1 Bert Classification Models

  • BERT and Covid-BERT classifiers were trained for up to 30 epochs with a 1.5e-5 learning rate and batch size 8.The BERT models had 345M parameters.Source: §A.3.1 · p. 16

A.3.2 Generative Models

  • Generation models used beam search with beam size 3 and were trained for up to 10 epochs, with early stopping based on development loss.GPT-2 used a 2e-5 learning rate and T5 used 5e-5; GPT-2 large used batch size 4, while other models used 16.Source: §A.3.2 · p. 16

A.4 Effect of Reader Perception on Article Sharing or Reading

  • Annotators generally rated real articles as more likely to be shared or read than misinformation, although 1.2% of misinformation articles in the training set received likelihood ratings of at least 4.Annotators generally said they would not share or read articles they thought were misinformation.Source: §A.4 · p. 16
  • Perceived reliability was positively related to sharing or reading, yet curiosity or a desire to corroborate could accompany sharing or reading misinformation despite skepticism.The authors give a headline about the coronavirus being intentionally released from a Wuhan lab as an example.Source: §A.4 · pp. 17–18

A.6 Further Related Work

  • MRF differs from rhetorical-framing formalisms based on techniques such as appeals to fear by modeling readers’ perceptions of writer intent, and goes beyond stance detection by representing how perceptions affect actions.The paper positions this focus as relevant to understanding how misinformation can be countered.Source: §A.6 · p. 18

A.5 Analysis of A/B Test

  • Generated writer-intent implications sometimes increased trust in real headlines or decreased trust in misinformation, but irrelevant implications and hallucinations could also reduce trust.The authors emphasize evaluating effects on both real news and misinformation.Source: §A.5 · p. 18, Table 12 · p. 19

Related papers

Loading 2104.08790v4…