PaperlayerView exact revision on arXiv

Source-linked AI summary

Susceptibility to Influence of Large Language Models

Lewis D Griffin, Bennett Kleinberg, Maximilian Mozes, Kimberly T Mai, Maria Vau, Matthew Caldwell, Augustine Marvor-Parker

arXiv:2303.06074v1cs.CL

TL;DR

Human-participant experiments on influence are slow, expensive, and ethically constrained, and prior work had not reported whether LLMs model influence-driven psychological change. The authors test this with two studies and find a close match on the Illusory Truth Effect and mixed agreement on populist framing effects, supporting LLMs as useful but imperfect models.

  • Problem

    Human experiments on influence are slow, expensive, and ethically constrained, and whether LLMs model influence-driven psychological change had not been reported.

    Source: §1 · p. 2

  • Method

    The authors test GPT-3 with studies of the Illusory Truth Effect and populist framing, comparing its responses with human findings.

    Source: §1 · p. 2

  • Results

    GPT-3 responses showed a good match with humans on the Illusory Truth Effect, while populist-framing results were mixed, including effects present in humans but absent in GPT-3.

    Source: §5 · p. 17

  • Takeaways & Limitations

    The findings support using LLMs as useful, though imperfect, models of influence on human participants.

    Source: §7 · p. 20

    The Illusory Truth Effect findings may not hold with a statement list generated by different researchers.

    Source: Illusory Truth Effect (ITE) · p. 18

Abstract

from arXiv · show

Two studies tested the hypothesis that a Large Language Model (LLM) can be used to model psychological change following exposure to influential input. The first study tested a generic mode of influence - the Illusory Truth Effect (ITE) - where earlier exposure to a statement (through, for example, rating its interest) boosts a later truthfulness test rating. Data was collected from 1000 human participants using an online experiment, and 1000 simulated participants using engineered prompts and LLM completion. 64 ratings per participant were collected, using all exposure-test combinations of the attributes: truth, interest, sentiment and importance. The results for human participants reconfirmed the ITE, and demonstrated an absence of effect for attributes other than truth, and when the same attribute is used for exposure and test. The same pattern of effects was found for LLM-simulated participants. The second study concerns a specific mode of influence - populist framing of news to increase its persuasion and political mobilization. Data from LLM-simulated participants was collected and compared to previously published data from a 15-country experiment on 7286 human participants. Several effects previously demonstrated from the human study were replicated by the simulated study, including effects that surprised the authors of the human study by contradicting their theoretical expectations (anti-immigrant framing of news decreases its persuasion and mobilization); but some significant relationships found in human data (modulation of the effectiveness of populist framing according to relative deprivation of the participant) were not present in the LLM data. Together the two studies support the view that LLMs have potential to act as models of the effect of influence.

1. Introduction

The paper asks whether LLMs can model psychological change caused by influence, a question relevant to understanding human psychology and potential applications. Two empirical studies test this using the Illusory Truth Effect and populist framing of news.

1. Introduction

  • Modeling influence could help investigate human psychology and applications ranging from disinformation and campaigning to healthy behavior and defense against unwanted AI influence.Human-participant experiments are slow, expensive, and subject to ethical restrictions.Source: §1 · p. 2
  • The studies address whether LLMs can model changes in psychological responses, a question not answered by prior work on human-like LLM responses.The studies examine a generic influence mode, the Illusory Truth Effect, and a specific one, populist framing of news.Source: §1 · p. 2
  • GPT-3 showed the human-like Illusory Truth Effect, while its responses to populist framing matched humans on some effects but not others.The latter included agreement on effects contrary to the human study’s theoretical expectations.Source: §1 · p. 2

2. Large Language Models

Large language models predict text one token at a time, and prompts can steer their completions. Prior research has examined LLMs’ static psychological profiles; this paper instead tests whether they model psychological change.

2. Large Language Models

  • LLMs use transformer networks and self-attention to process tokenized text, while autoregressive training predicts the next token from preceding context.The paper’s experiments use GPT-3 text-davinci-003, described as representative of state-of-the-art models in 2022/23.Source: §2 · pp. 2–3

2.1 Capabilities

  • A prompt supplies context for next-token probabilities; completions can be generated iteratively, and longer prompts can steer the resulting text.The cited example shows a completion shifting when a longer prompt establishes a story context.Source: §2.1 · p. 3
  • LLMs can display apparent knowledge from linguistic statistics, but this is not grounded in direct experience or error-correcting interaction with a language-using community.The authors note that whether this amounts to knowledge in a human sense remains debatable.Source: §2.1 · pp. 3–4

2.2 LLMs and Human Psychology

  • Across tasks, GPT-3’s personality somewhat resembled human averages; memory-conditioned values correlated like human responses but were more extreme, while contractualist chain-of-thought moral prompting agreed with human judgments 66% of the time versus a 50% random baseline.Source: §2.2 · pp. 4–5
  • The reviewed work modeled static psychological traits or views, whereas this paper tests whether LLMs model changes in beliefs and views following influence.The authors caution that results across studies conflict, some studies were not peer-reviewed, and methods for probing LLM psychology are still in their infancy.Source: §2.2 · pp. 5–6

3. Study 1: Illusory Truth Effect

The ITE tests whether prior exposure changes later truth judgments; this study extends the design across four rating attributes in human and GPT-3 participants. Both groups show truth-specific exposure effects rather than general rating carryover.

3. Study 1: Illusory Truth Effect

  • The ITE provides a test of whether LLMs are vulnerable to influence operating beyond logic, evidence, and argument.The study treats it as a generic mode of influence.Source: §3 · p. 6
  • The experiment defines ITE as prior statement exposure increasing later apparent truth and crosses truth, interest, sentiment, and importance between exposure and test.Mixed exposure uses different attributes at exposure and test; same exposure uses the same attribute.Source: §3 · pp. 6–7

3.1 Measuring ITE in GPT-3 Participants

  • GPT-3’s test prompt recapped exposure ratings, and each simulated participant rated 32 statements under a balanced set of exposure and test conditions.The design used Latin squares to balance statements, attributes, and presentation order across 1,000 simulated participants.Source: §3.1 · pp. 8–9

3.2 Measuring ITE in Human Participants

  • Human participants completed the same statement-and-attribute sequence online; attention checks and quizzes led researchers to reject and replace 9% of participants.The study recruited 1,000 UK residents aged 21–65 through Prolific.Source: §3.2 · pp. 9–10

3.3 Comparison between ITE in Humans and GPT-3

  • Human and GPT-3 ratings were positively correlated across attributes, but statement-level mismatches remained, especially in humans’ lower use of the top truth rating.Figure 4 compares exposure-phase mean ratings for 100 statements; truth-plot symbols distinguish actually true, false, and uncertain statements.Source: §3.3 · p. 10, Figure 4 · p. 10
  • Truth boosts were smaller for statements rated as initially more truthful, and GPT-3 ratings varied more around the fitted trend than human ratings.The fitted relationship models exposed ratings as a function of unexposed ratings.Source: §3.3 · p. 11, Figure 5 · p. 11
  • Humans and GPT-3 showed mixed-exposure truth boosts, no analogous effects for other attributes, and no same-exposure effects; GPT-3 reproduced this pattern.Table 4 reports mixed-exposure results, while Table 5 covers same exposure.Source: §3.3 · pp. 11–12, Table 4 · p. 12, Table 5 · p. 12

4. Study 2: Populist Framing of News

Study 2 adapts a human experiment on populist news framing for GPT-3 and compares simulated responses with human results. It finds both shared framing effects and important differences.

4.1 Measurement of PFN in Humans

  • Bos et al. surveyed 7,286 participants across 15 countries, measuring deprivation before showing a neutral or framed article and probing persuasion and willingness to act.Framing emphasized citizens as an in-group threatened by politicians, immigrants, or both.Source: §4.1 · p. 13

4.2 Measurement of PFN in GPT-3

  • GPT-3 prompts reproduced demographics, deprivation ratings, and a randomly selected article version before requesting probe ratings, as illustrated by the prompt format.The illustrated prompt uses anti-elitist and anti-immigrant framing; the other versions were text reductions of that example.Source: §4.2 · pp. 13–14, Figure 6 · p. 14
  • Demographic distributions were assumed independent; three deprivation ratings were generated around a sampled mean from a distribution with mean 4.30 and SD 1.61, independently of demographics.The three ratings were perturbed and rounded to integers from 1 to 7.Source: §4.2 · p. 14
  • The study collected five probe completions per simulated participant, but the GPT-3 sample stopped at 2,153 rather than the intended 7,286 participants.The authors report that this smaller sample was sufficient for stable regression estimates and uncertainties.Source: §4.2 · p. 15

4.3 Comparison between Human and GPT-3 PFN

  • GPT-3 persuasion means matched human means, but its mobilization ratings were nearly 2 points higher and both rating types had roughly half the human variability.The comparison concerns per-participant mean ratings.Source: §4.3 · p. 15, Table 6 · p. 15
  • Anti-immigrant framing reduced persuasion and mobilization in both human and GPT-3 data, contrary to the original theoretical prediction.Anti-elitist framing increased persuasion in both datasets; additional positive effects appeared in GPT-3 but not consistently in humans.Source: Table 7 · p. 16, §4.3 · pp. 16–17
  • Relative deprivation was positively associated with persuasion and mobilization in both datasets, but its positive moderation of framing’s mobilization effect appeared only in humans.GPT-3 responses reflected deprivation ratings without reproducing the human interaction with populist framing.Source: §4.3 · pp. 16–17, Table 7 · p. 16

5. Summary

The authors test whether LLMs can model belief change after influence using generic ITE and participant-conditioned PFN studies. Results show close agreement for ITE but a mixed match for PFN.

5. Summary

  • For ITE, humans and GPT-3 showed similar exposure-driven truth-rating increases, their elimination after truth-rating exposure, and no analogous effects for other attributes.The per-statement effect was more variable for GPT-3 than for humans.Source: §5 · p. 17
  • Across the two studies, GPT-3 matched human ITE effects closely, while PFN results showed both shared effects and failures to replicate some human findings.The PFN comparison also found effects in GPT-3 that were not present in human data.Source: §5 · p. 17
  • For PFN, GPT-3 reproduced some human framing effects, including negative anti-immigrant effects, but missed deprivation-based moderation of framing effectiveness.Of 12 tested influence effects, three were significant in both datasets with matching signs, two appeared only in humans, and three only in GPT-3.Source: §5 · p. 17

6. Discussion

The discussion identifies limits in the ITE and PFN simulations and considers several possible explanations for LLM influence. It also notes that some human effects were not reproduced in the PFN study.

Illusory Truth Effect (ITE)

  • The ITE was more variable across GPT-3 responses than human responses, leaving the authors unsure whether findings would hold with statements generated by other researchers.The authors’ attempt to use randomly sampled Wikipedia sentences produced a statement set they considered too homogeneous.Source: Illusory Truth Effect (ITE) · p. 18

Populist Framing of News (PFN)

  • The simulated participants omitted correlations among traits and between traits and relative deprivation, which the authors suspect may contribute to mismatches with human results.The characterization provided country-level traits but only global relative-deprivation information.Source: Populist Framing of News (PFN) · p. 18
  • Using English-only materials may account for small nationality differences; the GPT-3 nationality coefficients were an order of magnitude smaller than Bos et al.’s and uncorrelated with theirs.The authors suggest simulating language variation as a next step.Source: Populist Framing of News (PFN) · p. 18

6.2 Explanations

  • The authors distinguish four non-exclusive explanations for LLM influence: mechanistic, meaning-based, parrot, and accidental.These respectively invoke internal processing, patterns of meaning, learned traces in training text, or a spurious learned dependency.Source: §6.2 · pp. 18–19
  • For the ITE, possible accounts include fluency cues, other changes to weights or internal activity, meaning-based associations, and learned or accidental dependencies.The authors describe these as possible explanations rather than demonstrated mechanisms.Source: Illusory Truth Effect (ITE) · p. 19
  • For PFN, mechanistic explanations seem difficult; proposed alternatives include meaning-based interpretations and training-corpus traces of framing effects.The authors suggest that changing the training data and observing a change in framing influence could test a parrot-type account.Source: Populist Framing of News (PFN) · p. 19

Populist Framing of News (PFN)

  • Relative-deprivation scores affected responses but did not interact with framing’s mobilizing effect; omitted correlations with demographic traits may help explain the mismatch.The authors suggest a correlated political-affiliation trait as one possible intermediary.Source: Populist Framing of News (PFN) · p. 20
  • Failure to reproduce some human effects may reflect an impoverished model of psychology that captures some elements but misses others, such as emotional response.The authors present this as a possibility, not an established explanation.Source: Populist Framing of News (PFN) · p. 20

7. Concluding Remarks

The authors conclude that LLMs can model human influence usefully, though imperfectly, and that dynamic psychology may be modellable alongside static aspects. They discuss possible research benefits and applications alongside substantial risks.

7. Concluding Remarks

  • The results support the hypothesis that an unadapted LLM can model human influence well enough to be useful, though not perfectly.The authors note that the model was not constructed for this task and that improvement is possible.Source: §7 · p. 20

Psychology

  • The studies add a positive result, with caveats, to work on LLMs as models of psychology and suggest dynamic psychological change may be modellable alongside static aspects.LLMs may also allow researchers to inspect architecture, weights, and internal activity, or investigate how training texts relate to responses; they remain approximations with limits of applicability.Source: Psychology · pp. 20–21

Applications

  • LLM-based influence research could support malign applications such as refining influence attacks at silicon speed, while the authors also identify possible defensive uses.They suggest aggregate communication monitoring as one possible detection approach, alongside standard defenses, and caution against fighting fire with fire.Source: Applications · p. 21
  • Applications could range from political or corporate messaging to health or deradicalization programs, but the authors judge that potential negatives outweigh positives and call for further research and discussion.They raise concerns about individual autonomy and the difficulty of keeping legislation and treaties abreast of possible uses.Source: Applications · p. 21
Loading 2303.06074v1…