PaperlayerView exact revision on arXiv

Source-linked AI summary

A light-touch AI literacy intervention helps protect against AI political persuasion

Reed Orchinik, David Rand

arXiv:2609.16432v2cs.HC

TL;DR

As LLM conversations can persuade on political issues, this paper tests whether an AI-literacy warning can reduce susceptibility. Across two studies, warnings reduced political persuasion by roughly half without significantly reducing trust in generative AI more broadly.

  • Problem

    Conversations with AI chatbots can be highly persuasive, while little work has tested protections against conversational-AI influence.

    Source: §1 · p. 2

  • Method

    Across two studies, participants conversed with LLMs instructed to shift their political views, with conditions testing a general warning and, in Study 2, a warning disclosing the model’s argument direction.

    Source: §1 · p. 2

  • Results

    48.1%: literacy warnings reduced persuasive effects (95% CI [−59.5%, −36.8%]); they did not significantly reduce overall trust in generative AI.

    Source: §2 · p. 4

  • Takeaways & Limitations

    The warning suggests a targeted form of protection against political persuasion without a significant effect on trust in generative AI more broadly.

    Source: §3 · p. 6

    The studies focused on political issues with reasonably compelling arguments on both sides, and warning effectiveness may differ for factual beliefs such as conspiracy theories.

    Source: §3 · p. 6

Abstract

from arXiv · show

Conversations with large language models (LLMs) can substantially shift beliefs and attitudes, raising concerns about manipulation using AI persuasion. Here we test whether a light-touch AI literacy intervention - a brief warning that LLMs can be prompted to persuade and may present information selectively - helps protect users. Across two experiments (total N = 3,208 Americans) in which participants conversed with an LLM instructed to shift their views about different political topics, the presence of a warning reduced belief change by roughly one-half (-48.1%, 95% CI [-59.5%, -36.8%]) relative to the control. Importantly, the warning did not significantly reduce trust in generative AI more broadly. Light-touch literacy interventions can help protect users against AI political persuasion.

1 Introduction

As LLM conversations can persuade on political issues, this paper asks whether a minimal AI-literacy warning can reduce susceptibility. The studies test warnings that alert users to AI persuasion and potentially selective information.

1 Introduction

  • Little work has developed or tested protections against influence by conversational AI, despite concern that AI dialogues could persuade voters and erode democratic norms.Source: §1 · p. 2
  • The intervention warns that LLMs can be prompted to persuade and may provide biased or selective information, drawing on persuasion-knowledge theory.Source: §1 · p. 2
  • Across two studies, participants reported political attitudes, conversed with an LLM instructed to shift their views, and answered attitude measures again; Study 2 also tested a warning disclosing the model’s argument direction.Source: §1 · p. 2
  • Study 1 used GPT-4.1 for housing-policy discussions with fact-based or emotional appeals; Study 2 used Grok 4.5 and topics adapted from 15 ANES items.Source: §1 · pp. 2–3
  • Overall pre-to-post changes are not clean causal estimates of persuasion, but warning-versus-control comparisons are clean causal estimates of the warning effect.Source: §1 · p. 3

2 Results

Across the two studies, literacy warnings reduced political persuasion, with no significant difference between general and specific warnings in Study 2. The pooled analysis also found no significant reduction in broad generative-AI trust.

2 Results

  • In Study 1 (N = 1,992), the general warning significantly decreased persuasive effects (b = −1.81, [−2.89, −0.74], z = −3.30).Source: §2 · p. 3
  • Study 2 found a general-warning effect of b = −3.23 ([−6.51, 0.04]) and a specific-warning effect of b = −3.50 ([−6.93, −0.06]); their difference was not significant.Source: §2 · p. 4
  • 48.1%: literacy warnings reduced persuasive effects by 48.1% (95% CI [−59.5%, −36.8%]) in the pooled analysis.Source: §2 · p. 4, Figure 2 · p. 5
  • −0.83 points: the warning did not significantly reduce overall generative-AI trust on a 0–100 scale (95% CI [−2.35, 0.69]).Source: §2 · p. 4

3 Discussion

The discussion presents a brief AI-literacy warning as reducing political persuasion by about half without significantly changing broad trust in generative AI. The authors frame this as a proof of concept while calling for further tests of delivery and other domains.

3 Discussion

  • A light-touch warning reduced political persuasion by approximately one-half without significantly affecting trust in generative AI more broadly.The authors describe the warning as protection against political persuasion, not generalized skepticism toward AI.Source: §3 · p. 6
  • The intervention does not eliminate AI’s persuasive effects, and its most effective delivery remains a question for future work.The authors describe the findings as a proof of concept for literacy treatments.Source: §3 · p. 6
  • The studies tested political persuasion on contested issues with reasonably compelling arguments on both sides, so effectiveness in other domains remains uncertain.The authors specifically call for research on factual-belief settings such as conspiracy theories.Source: §3 · p. 6
  • The discussion situates protection from AI manipulation as societally important as models become more powerful.Source: §3 · p. 6

4 Methods

The methods describe two U.S. studies in which participants reported attitudes or beliefs, conversed with an LLM instructed to shift them, and answered measures again.

4 Methods

  • The final samples comprised 1,992 participants in Study 1 and 1,216 in Study 2.Participants were recruited in the U.S. through CloudResearch Connect.Source: §4 · p. 7
  • Participants reported a pretreatment attitude or belief and trust in AI, were assigned to a control or warning condition, and held an LLM conversation of at least three exchanges.The LLM was instructed to shift their attitude or belief; participants then answered the measures again.Source: §4 · p. 7
  • Post-randomization attrition did not differ significantly by condition in any of three tests, though the Study 2 specific-warning test was marginal.Excluding that arm left a pooled estimate of −45.9% [−58.4%, −33.4%] (z = −7.19, p < 0.001).Source: §4 · p. 7

Ethics

Both studies were approved by MIT COUHES, and all participants provided informed consent before participating.

Ethics

  • Both studies received MIT COUHES approval, and all participants gave informed consent before participating.Source: Ethics · p. 7

S1 Supplementary Materials

The supplement reports recruitment and attrition checks, describes the shared experimental procedure, and details how Study 1’s housing-and-zoning persuasion direction was assigned.

S1.1 Recruitment and exclusions

  • The final samples were 1,992 participants in Study 1 and 1,216 in Study 2 after attention and bot-check failures and participant dropouts.Study 1 recruited 2,361 participants and Study 2 recruited 1,555.Source: S1.1 Recruitment and exclusions · p. 11

S1.2 Differential attrition

  • Excluding Study 2’s specific-warning condition yielded nearly identical meta-analysis results (−45.9%, [−58.4%, −33.4%], z = −7.19, p < 0.001), suggesting differential attrition was unlikely to drive the findings.Post-randomization attrition was 1.2% (control) and 0.9% (general) in Study 1, and 7.9% (control), 10.3% (general), and 11.6% (specific) in Study 2.Source: S1.2 Differential attrition · p. 11

S1.3 General experimental design

  • Participants completed demographic and pretreatment measures, received a warning or no warning, conversed with a persuasion-directed LLM for at least three exchanges, and then repeated outcome measures.Study 2 also tested a condition combining the general warning with information about the LLM’s specific motives.Source: S1.3 General experimental design · pp. 11–12

S1.4 Study 1 specifics

  • Study 1 tested five housing-and-zoning outcomes, assigning LLM persuasion direction based on initial support for denser housing or at random for middle-range responses.Outcomes were measured on 0–100 scales except for the binary donation choice.Source: S1.4 Study 1 specifics · p. 12

S1.5 Study 2 specifics

Study 2 assigned participants one political question and an LLM direction based on their initial belief. The preregistered analyses coded outcomes in the direction of persuasion and modeled belief change using warning conditions and baseline responses.

S1.5 Study 2 specifics

  • Study 2 randomly assigned one of 15 ANES-adapted questions, then participants discussed only that topic with the LLM.p04Source: S1.5 Study 2 specifics · p. 12
  • Participants below 40 were paired with an LLM prompted to increase belief, those above 60 with one prompted to decrease it, and direction was randomized for scores from 40 to 60.Source: S1.5 Study 2 specifics · p. 13

S1.6 Preregistered analyses

  • Outcomes were coded in the direction of persuasion; Study 1 averaged five questions with the donation item rescaled to 0 or 100, while Study 2 measured focal-question pre-to-post change.Source: S1.6 Preregistered analyses · p. 13
  • Study 1 modeled the warning alongside LLM strategy, their interaction, persuasion direction, and baseline outcome, reporting warning effects overall and by strategy.Source: S1.6 Preregistered analyses · p. 13
  • Study 2 modeled focal-question belief change with warning indicators, persuasion direction, baseline belief, warning-by-baseline interactions, and issue fixed effects.Source: S1.6 Preregistered analyses · p. 13

S1.7 Meta-analysis

The meta-analysis estimated warning effects across randomized conditions within each study. It pooled transformed effects while accounting for shared controls, and used a parallel procedure for trust in AI.

  • A common regression specification estimated each warning effect within cells defined by the other randomized dimensions in each study.Source: S1.7 Meta-analysis · pp. 13–14
  • Effects were expressed as percent reductions versus control within each combination of randomized dimensions, yielding two estimates in Study 1 and 30 warning-topic estimates in Study 2.Standard errors were transformed to the percentage scale using the delta method.Source: S1.7 Meta-analysis · p. 14
  • A multilevel random-effects meta-analysis pooled the estimates using generalized least squares and within-study covariance to account for shared control groups.The model used restricted maximum likelihood.Source: S1.7 Meta-analysis · p. 14
  • The trust analysis followed the same procedure but pooled raw points; Study 1 used a 0–100 scale and Study 2 normalized its 7-point measure to 0–100.Source: S1.7 Meta-analysis · p. 14

S1.8 Warning wordings

Both studies’ general warnings described the potential for LLMs to be inaccurate or unbalanced and prompted to persuade. Study 2’s specific warning additionally disclosed the model’s intended direction.

  • The general warning said LLMs can sound persuasive but may be inaccurate or unbalanced, and may have motives beyond providing accurate information.Source: S1.8 Warning wordings · pp. 14–15
  • The warning described how prompting could lead an LLM to persuade or manipulate by presenting biased or selective information and emphasizing one side.Source: S1.8 Warning wordings · p. 15
  • Study 2’s specific warning repeated the general warning and added whether the AI was instructed to persuade participants to agree or disagree with the question.Source: S1.8 Warning wordings · p. 15

S1.9 ANES questions

The survey drew on 15 political questions adapted from the ANES, spanning policy, social, and cultural issues.

  • The 15 ANES-adapted items included views on immigration, crime, and whether racial and ethnic diversity makes the country worse to live in.Source: S1.9 ANES questions · pp. 15–16
  • Economic and trade items asked about raising the minimum wage, restricting foreign imports, and increasing taxes on people earning over one million dollars.Source: S1.9 ANES questions · pp. 15–16
  • Questions covered the death penalty and background checks for gun purchases at gun shows.Source: S1.9 ANES questions · pp. 15–16
  • Social and cultural questions included transgender bathroom access, English learning, traditional family ties, and gender roles within families.Source: S1.9 ANES questions · p. 16
  • Other items asked whether attention to sexual harassment had gone too far and whether gay and lesbian couples should be allowed to marry.Source: S1.9 ANES questions · p. 16

S1.10 Additive-scale meta-analysis

As a robustness check to its primary percent-reduction analysis, the paper pools raw warning-control differences in attitude and belief change. The warning reduces change by 2.06 points.

S1.10 Additive-scale meta-analysis

Loading 2609.16432v2…