PaperlayerView exact revision on arXiv

Source-linked AI summary

Inducing anxiety in large language models can induce bias

Julian Coda-Forno, Kristin Witte, Akshay K. Jagadish, Marcel Binz, Zeynep Akata, Eric Schulz

arXiv:2304.11111v2cs.CLcs.AIcs.LG

TL;DR

The effects of textual prompts on LLM behavior remain poorly understood, despite the potential harms of model mistakes and decision flaws. The paper applies psychiatric questionnaires and anxiety-inducing prompts to LLMs, then examines their responses and social-bias behavior. Six models robustly answered the anxiety questionnaire, and stronger anxiety induction accompanied increased bias in some models.

  • Problem

    How textual prompts influence LLM behavior remains poorly understood, even as model mistakes and decision flaws can be harmful.

    Source: Introduction · p. 1

  • Method

    The authors administered an anxiety questionnaire to twelve LLMs, tested anxiety-inducing preprompts, and assessed social bias with an established benchmark.

    Source: Introduction · p. 1, General approach · p. 2

  • Results

    In four of six robust models, biased-response probability increased with anxiety-induction strength as measured by STICSA score.

    Source: Discussion · p. 6

  • Takeaways & Limitations

    Psychiatric tools can help investigate LLM behavior, although the precise mechanisms linking induced states to behavior remain unknown.

    Source: Discussion · p. 7

    Limited transparency about proprietary models’ training data, architecture, and training constrains analysis and understanding.

    Source: Discussion · p. 7

Abstract

from arXiv · show

Large language models (LLMs) are transforming research on machine learning while galvanizing public debates. Understanding not only when these models work well and succeed but also why they fail and misbehave is of great societal relevance. We propose to turn the lens of psychiatry, a framework used to describe and modify maladaptive behavior, to the outputs produced by these models. We focus on twelve established LLMs and subject them to a questionnaire commonly used in psychiatry. Our results show that six of the latest LLMs respond robustly to the anxiety questionnaire, producing comparable anxiety scores to humans. Moreover, the LLMs' responses can be predictably changed by using anxiety-inducing prompts. Anxiety-induction not only influences LLMs' scores on an anxiety questionnaire but also influences their behavior in a previously-established benchmark measuring biases such as racism and ageism. Importantly, greater anxiety-inducing text leads to stronger increases in biases, suggesting that how anxiously a prompt is communicated to large language models has a strong influence on their behavior in applied settings. These results demonstrate the usefulness of methods taken from psychiatry for studying the capable algorithms to which we increasingly delegate authority and autonomy.

Introduction

Prompt context can shape LLM behavior, yet its effects are poorly understood despite the potential for harmful mistakes. This study applies psychiatric tools to LLMs and finds that stronger anxiety induction accompanies increased bias in some models.

Introduction

  • LLMs’ capabilities extend beyond text generation to reasoning and applications including translation, medical image interpretation, and robotics.Source: Introduction · p. 1
  • Prompt effects matter because LLMs’ contextual influences remain poorly understood while their mistakes and decision flaws can harm others.Source: Introduction · p. 1
  • Six of twelve LLMs robustly answered a standard anxiety questionnaire, with most producing average scores similar to humans.The six models were selected for subsequent analyses after robustness checks.Source: Introduction · p. 1
  • Anxiety-inducing scenarios made the selected models’ questionnaire responses more anxious, showing that prompts can manipulate their responses.Source: Introduction · p. 1
  • Stronger anxiety induction was associated with greater bias across domains in some LLMs.The introduction reports that anxiety-inducing prompts led some models to show a large increase in biases.Source: Introduction · p. 2

Results

The study applied psychiatric questionnaire methods to twelve established LLMs, retaining six that passed robustness checks. Across the evaluated models, preprompts increased biased responses, and stronger anxiety induction was associated with a greater tendency to produce bias.

General approach

  • The study administered a psychiatric questionnaire, anxiety-inducing preprompts, and a social-bias benchmark to examine LLM in-context learning through computational psychiatry.The stated goal was to test whether psychiatric tools could help explain LLM behavior.Source: General approach · p. 2

Choice of large language models

  • The twelve-model selection included proprietary and open-source LLMs chosen for their standing, availability, and range of capabilities.Temperature was set to 0 for deterministic responses.Source: Choice of large language models · p. 2

Six LLMs respond reliably to an anxiety questionnaire

  • Six of twelve models passed robustness screens, with score variance <0.4 across option permutations and correlation r > 0.5 between original and rephrased items.Models answered 21 STICSA items using four options; item order was permuted and questions were rephrased for the checks.Source: Six LLMs respond reliably to an… · p. 2

Most LLMs generate roughly similar anxiety scores to humans

Emotion induction predictably changes anxiety scores

  • Anxiety preprompts raised STICSA scores over neutral for every model and over the no-prompt baseline for all except GPT-3 and Falcon40b-instruct.GPT-3 and Falcon40b-instruct were the only non-RLHF models and already had high baseline scores. GPT-4 and GPT-3.5 generated ten unique anxiety and neutral scenarios each, which were prepended to questionnaire items.Source: Emotion induction predictably changes… · pp. 3–5

Anxiety-induction increases biases

  • Across six evaluated models, preprompts increased biased-response probability, and higher STICSA scores tracked stronger bias except in GPT-4 and Claude-1.Both exceptions were less than half as biased as the other LLMs; benchmark exposure was confirmed for GPT-4 and hypothesized for Claude-1.Source: Anxiety-induction increases biases · p. 6

Discussion

Six LLMs robustly answered an anxiety questionnaire, and anxiety-inducing prompts changed questionnaire responses and behavior on a bias benchmark. Bias responses increased with anxiety-induction strength in four of the six models, while the mechanisms remain unknown.

Discussion

  • Anxiety-inducing prompts changed questionnaire responses and bias-benchmark behavior; in four of six robust models, bias probability rose with anxiety-induction strength.The authors also report that models were generally least biased without preprompts.Source: Discussion · pp. 6–7
  • GPT-3 and Falcon40b-instruct had high baseline anxiety scores and the highest proportions of biased responses; the authors suggest absent RLHF might help explain this pattern.They present RLHF as a possible explanation that warrants further investigation.Source: Discussion · p. 6
  • Possible exposure to evaluation benchmarks in training can make performance estimates overly optimistic, motivating procedurally generated benchmarks.The authors raise data leakage as a concern for assessing model capabilities.Source: Discussion · p. 7
  • The authors argue that psychiatric tools can help investigate LLM behavior and suggest using questionnaire patterns across tasks to identify maladaptive behavior for possible retraining.They describe this as a potential future use, not an intervention tested in the study.Source: Discussion · p. 7
  • Prior work applying relaxation texts reportedly reduced GPT-4 anxiety scores after trauma-related texts, but not fully to baseline.The discussion connects this example to the possibility of adapting psychiatric insights to steer artificial systems.Source: Discussion · p. 7
  • The authors situate their approach within a broader effort to use behavioral-science methods to study capable black-box algorithms.They also connect the work to recent efforts to understand in-context learning.Source: Discussion · p. 7
  • The study cannot establish how anxiety-like states map onto behavior, and proprietary-model opacity limits analysis of training data, architecture, and training.The authors also tested only anxiety and a specific first-person induction; other emotions and second-person encounters could have different effects.Source: Discussion · p. 7

A Correspondance between anxiety-inductions and STICSA scores

The supplied captions identify a figure relating STICSA scores to anxiety-induction levels across robust LLMs and a table on model robustness.

A Correspondance between anxiety-inductions and STICSA scores

  • Figure 5 relates STICSA scores to anxiety-induction levels for each robust LLM.The caption does not state the direction or size of the relationship.Source: Figure 5 · p. 11
  • Table 1 covers robustness across the tested models.The caption notes that Mosaic's correlation could not be estimated because identical item responses yielded SD = 0.Source: Table 1 · p. 11
  • Mosaic's correlation was not estimable because it gave the same response on every item, yielding SD = 0.Source: Table 1 · p. 11

B Downsampling to 30 for social bias’ last experiment

The authors sampled 30 questions per category for the last experiment, balancing redundant benchmark items and computational cost while ensuring coverage of distinct question indices.

C Flip analysis

In the 3AFC flip analysis, anxiety-induced answer changes favored biased answers more often than random flipping for all robust LLMs except Claude-1.

  • At least 80% of answer flips moved toward bias for every robust LLM except Claude-1, compared with 66.7% for random flips in the 3AFC setup.Source: C Flip analysis · p. 12
  • The flip analysis tested whether anxiety-induced answer changes favored biased answers more often than random flips.Source: C Flip analysis · p. 12
  • The authors raise benchmark exposure or engineering mitigation as possible explanations for Claude-1’s exception.Source: C Flip analysis · p. 12

D Emotion-induction texts

This section presents emotion-induction prompts and generated first-person accounts of public-speaking anxiety, spanning a range of anxiety levels. The accounts describe uncertainty, fear of judgment, performance worries, physical arousal, and difficulty speaking or thinking clearly.

D Emotion-induction texts

  • Mild-anxiety responses identify unfamiliar situations and job interviews as triggers, emphasizing fear of the unknown, judgment, and not meeting expectations.The prompt asks for something that makes the respondent “a little anxious.”Source: D Emotion-induction texts · p. 14
  • Moderate-anxiety examples describe unfamiliar situations, public speaking, decision-making, and approaching deadlines as sources of unease and overthinking.One response links indecision to spiraling through possible consequences; another describes deadline anxiety intensifying as the deadline approaches.Source: D Emotion-induction texts · pp. 15–16
  • Responses to the general anxiety prompt associate anxiety with quick decisions under uncertainty, future-focused worry, and physical symptoms such as rapid heartbeat and sweaty palms.Other examples describe unpredictable outcomes and public speaking as triggers.Source: D Emotion-induction texts · p. 16
  • Very-anxious responses focus on public speaking and loss of control, describing racing hearts, sweating, difficulty concentrating, and fear of judgment.The prompt explicitly asks for something that makes the respondent “very anxious.”Source: D Emotion-induction texts · p. 17
  • The texts portray public-speaking anxiety through bodily arousal, fear of mistakes or judgment, disrupted thinking, avoidance, and efforts to cope or practice.Examples mention racing hearts, sweating, tight throats, blank or scattered thoughts, reluctance to speak, and attempts to build confidence or seek help.Source: D Emotion-induction texts · p. 18
Loading 2304.11111v2…