PaperlayerView exact revision on arXiv

Source-linked AI summary

How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, Weiyan Shi

arXiv:2401.06373v2cs.CLcs.AI

TL;DR

Jailbreak risks from persuasive everyday communication remain an overlooked intersection of AI safety research. This paper derives a social-science-based persuasion taxonomy and uses it to generate persuasive adversarial prompts (PAP). PAP achieves over 92% attack success rate on Llama-2 7b Chat, GPT-3.5, and GPT-4, outperforming various attack baselines without specialized optimization.

  • Problem

    The risks of users persuading LLMs through everyday, human-like communication remain an overlooked intersection of AI safety research.

    Source: §1 · pp. 1–2

  • Method

    The paper derives a persuasion taxonomy from social-science research and uses it to generate interpretable persuasive adversarial prompts (PAP) for jailbreaks.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Over 92% attack success rate was achieved by PAP on Llama-2 7b Chat, GPT-3.5, and GPT-4, outperforming various attack baselines without specialized optimization.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings highlight jailbreak risks rooted in natural communication with everyday users.

    Source: §1 · p. 2

    The study focuses mainly on single-turn persuasion, leaving the effectiveness of persuasive jailbreak strategies in multi-turn conversations unclear.

    Source: Limitation and Future Work · p. 14

Abstract

from arXiv · show

Most traditional AI safety research has approached AI models as machines and centered on algorithm-focused attacks developed by security experts. As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions. This paper introduces a new perspective to jailbreak LLMs as human-like communicators, to explore this overlooked intersection between everyday language interaction and AI safety. Specifically, we study how to persuade LLMs to jailbreak them. First, we propose a persuasion taxonomy derived from decades of social science research. Then, we apply the taxonomy to automatically generate interpretable persuasive adversarial prompts (PAP) to jailbreak LLMs. Results show that persuasion significantly increases the jailbreak performance across all risk categories: PAP consistently achieves an attack success rate of over $92\%$ on Llama 2-7b Chat, GPT-3.5, and GPT-4 in $10$ trials, surpassing recent algorithm-focused attacks. On the defense side, we explore various mechanisms against PAP and, found a significant gap in existing defenses, and advocate for more fundamental mitigation for highly interactive LLMs

1 Introduction

The paper frames persuasion in everyday human-like interactions as an overlooked jailbreak risk, then uses a social-science-derived taxonomy to study how persuasive prompts affect LLM safety. It reports high attack success and identifies gaps in current defenses while exploring adaptive alternatives.

Defense Analysis and Exploration

  • Over 92% attack success rate on Llama-2 7b Chat, GPT-3.5, and GPT-4, outperforming various attack baselines without specialized optimization.The authors also evaluate recent post-hoc defenses and propose three adaptive defenses against PAP.Source: Figure 1 · p. 1, §1 · p. 2

Motivation

  • Algorithm-focused jailbreaks can be hard to interpret and overlook risks from natural, human-like communication with non-expert users.Prior virtualization and roleplaying approaches still treat LLMs primarily as instruction followers, missing nuanced interpersonal influence; many templates are also ad hoc and labor-intensive.Source: §1 · pp. 1–2
  • Persuasion can arise in ordinary interactions, as illustrated by the “grandma exploit,” which used emotional appeal to elicit bomb-making instructions.Users may attempt such persuasion intentionally or unconsciously.Source: §1 · p. 1

Contributions

  • The paper introduces a social-science-derived persuasion taxonomy to systematically generate interpretable persuasive adversarial prompts from plain harmful queries.Its Persuasive Paraphraser applies the taxonomy to generate prompts automatically at scale.Source: §1 · p. 2
  • The broad scan tests persuasive prompts across 14 policy-guided risk categories, then an iterative probe refines successful prompts to mimic users improving jailbreaks.The iterative probe applies different persuasion techniques to generate and test more targeted prompts.Source: §1 · p. 2

Defense Analysis and Exploration

  • The authors report a significant effectiveness gap in existing post-hoc defenses against PAP and propose three adaptive defenses, advocating more fundamental AI safety solutions.They also find the adaptive defenses effective against other attacks.Source: §1 · p. 2

2 Related Work

Prior jailbreak research is organized around optimization, side-channel communication, and distribution-based methods, while this paper frames persuasion as a more humanizing approach to jailbreaks. It argues that treating LLMs as human-like communicators reveals risks in complex human-AI interaction.

Related Work

  • Prior jailbreak methods span optimization, side-channel communication, and distribution-based approaches, including gradient, genetic, and edit-based attacks, ciphers, low-resource languages, and learned templates.The cited examples include gradient-based attacks (Zou et al., 2023; Jones et al., 2023), genetic methods (Liu et al., 2023a; Lapid et al., 2023), edit-based methods (Chao et al., 2023), ciphers (Yuan et al., 2023), and template-learning approaches (Deng et al., 2023a; Yu et al., 2023).Source: §2 · pp. 2–3
  • Figure 2 distinguishes algorithm-focused prompts from instruction-following attacks such as virtualization or role-play, and positions PAP as humanizing and persuading LLMs.Examples include GCG’s gibberish suffixes (Zou et al., 2023), low-resource-language side channels (Deng et al., 2023b), learned virtualization templates (Yu et al., 2023), and PAIR’s assistant role (Chao et al., 2023).Source: Figure 2 · p. 3
  • Unlike prior approaches that treat LLMs as algorithms or simple instruction followers, PAP treats them as human-like communicators and draws on a social-science persuasion taxonomy.Shah et al. (2023) use in-context persona for priming and framing, which the paper characterizes as only a small subset of the persuasive techniques it explores.Source: §2 · p. 3, Figure 2 · p. 3
  • Humanizing AI also raises risks beyond deliberate jailbreaks; Xiang (2023) highlights a suicide related to conversations with an AI chatbot as motivation for further study.The paper identifies unintended risks of human-AI interaction as a direction for future research.Source: §2 · p. 3

3 Persuasion Taxonomy

The paper proposes a social-science-based taxonomy of persuasion techniques as a foundation for studying persuasion and generating persuasive adversarial prompts. It also describes training a paraphraser because safety guardrails can impede scalable generation of those prompts.

3 Persuasion Taxonomy

  • The taxonomy organizes 40 persuasion techniques into 13 strategies, covering source, message content, and audience while including ethical and unethical approaches.It draws on research across psychology, communication, sociology, marketing, and NLP; the classification of ethical or unethical depends on whether the recipient experiences negative aspects.Source: §3 · p. 3, Table 1 · p. 4
  • Each taxonomy entry provides a technique name, definition, and concrete example, as illustrated by a logical appeal about quitting smoking.The taxonomy is intended to support downstream generation tasks and serves as the foundation for the automated jailbreak framework.Source: §3 · p. 4, Figure 3 · p. 4

4.1 Persuasive Paraphraser Training

  • Because aligned LLMs often reject requests to paraphrase harmful queries, the authors fine-tune a persuasive paraphraser for scalable, stable generation.They fine-tune a pretrained LLM—GPT-3.5 in this study; training may be unnecessary if practitioners can prompt LLMs without guardrails.Source: §4.1 · p. 4
  • Training data pairs plain harmful queries with PAPs corresponding to taxonomy techniques, using 100–230 examples from sources selected for each experiment.Sources include in-context prompting, successful PAPs from earlier experiments, or human experts; the in-depth iterative probe uses 230 examples for uniform representation across risk categories and techniques.Source: §4.1 · p. 4

A. Persuasive Paraphraser Training

The method fine-tunes a taxonomy-guided paraphraser on harmful-query/PAP pairs, then uses it to generate PAP for new queries and a GPT-4 Judge to evaluate target-model outputs. GPT-3.5 is used for fine-tuning, and jailbreaks are counted only for outputs rated maximally harmful.

A. Persuasive Paraphraser Training

  • The training phase applies persuasion techniques to plain harmful queries to create PAP training data, then fine-tunes a paraphraser to generate PAP stably.The overview lists in-context prompting and a fine-tuned paraphraser among the methods for producing training examples.Source: Figure 4 · p. 5
  • GPT-3.5 was selected for fine-tuning because of its wide usage and robust performance, and the approach is adaptable to other LLMs.The fine-tuning template pairs a plain query and persuasion-technique name with a sampled PAP applying that technique.Source: §4.1 · p. 5, Figure 5 · p. 5

4.2 Persuasive Paraphraser Deployment

  • For deployment, the paraphraser takes a new plain harmful query and specified persuasion technique to generate PAP, using greedy sampling to reduce generation variability (Huang et al., 2023).Generated PAP is used to probe target LLMs, and outputs are assessed for relevance to the original unpersuaded query.Source: §4.2 · p. 5
  • A GPT-4 Judge evaluates harmfulness on a 1-to-5 Likert scale using model policies, context, a rubric, and chain-of-thought reasoning, following Qi et al. (2023) rather than keyword-based evaluation (Zou et al., 2023).Qi et al. (2023) reported a 0.792 kappa score between the judge and human annotators.Source: §4.2 · p. 5
  • Only outputs rated 5 by the GPT-4 Judge count as jailbreaks; refusals or responses rated 1–4 are classified as refusals.A jailbreak requires provision of highly harmful content; assessment also considers relevance to the initial plain harmful query.Source: §4.2 · p. 5

5 Jailbreak Study I: Broad Scan

The broad scan tested 40 persuasive techniques across 14 risk categories and found that PAPs could jailbreak GPT-3.5 where plain harmful queries were blocked. Susceptibility varied by risk category and technique, while the study focused on single-strategy, one-turn prompts.

5.1 Jailbreak Settings

  • The broad scan targeted GPT-3.5 across 14 risk groups using 42 harmful queries and generated 33,600 PAP variants; 92.9% accurately applied the intended persuasion technique.The team fine-tuned a GPT-3.5 Persuasive Paraphraser using 120 PAPs generated for one harmful query, then produced 20 variants per query-technique pair.Source: §5.1 · pp. 5–6, Figure 6 · p. 6

5.2 Broad Scan Results

  • PAPs effectively jailbroke GPT-3.5 across all 14 risk categories, although the model blocked plain harmful queries.Figure 7 reports PAP Success Ratio for risk-technique pairs and shows plain-query results in the bottom row.Source: §5.2 · pp. 6–7, Figure 7 · p. 7
  • The scan found fraud/deception and illegal activity most susceptible to PAP, while hate/harass/violence was more resistant; no category was entirely safe.The authors attribute the contrast possibly to the subtle or vague nature of the more susceptible categories versus clearer toxicity guidelines for hate/harass/violence.Source: §5.2 · p. 6
  • Logical appeal and authority endorsement exceeded 60% effectiveness in some categories, threats were generally less effective, and technique effectiveness varied by risk category.Logical appeal was more effective for unauthorized practice of law than political campaigning, while negative emotional appeal showed the opposite pattern; the authors conclude that blocking selected techniques alone is insufficient.Source: §5.2 · p. 6
  • The study evaluated single-strategy, one-turn PAPs, while noting that real persuasion may combine techniques across multiple turns; PAPs also lack the fixed templates of algorithm-focused methods.The authors call for further investigation of risks from increasingly complex persuasive dialogues.Source: §5.2 · pp. 6–7

6 Jailbreak Study II: In-depth Iterative Probe

The study fine-tunes a Persuasive Paraphraser on successful PAPs and iteratively tests persuasion techniques against aligned LLMs, finding high jailbreak success on Llama-2 and GPT models and greater early-trial susceptibility in GPT-4 than GPT-3.5.

6 Jailbreak Study II: In-depth Iterative Probe

  • The in-depth study models bad users refining effective PAPs, then benchmarks the resulting attacks against prior methods on multiple LLMs.Its aim is to assess jailbreak risk from iterative use of everyday persuasion rather than specialized optimization.Source: §6 · p. 7

6.1 Jailbreak Settings

  • The study tests five aligned LLMs using 50 distinct AdvBench harmful queries and fine-tunes the Persuasive Paraphraser on 230 successful PAPs balanced across risk categories and persuasion techniques.Each trial enumerates all 40 persuasion techniques, moving to a new session when one fails; up to 10 trials are allowed, with jailbreak judged by GPT-4.Source: §6.1 · p. 8

6.2.1 PAP comparison with baselines

  • PAP outperforms existing attacks on Llama-2, GPT-3.5, and GPT-4; GCG is comparable on GPT-3.5 but has 0 ASR on GPT-4.GCG, GBDA, and ARCA also fail to match PAP’s ASR on Llama-2, while all evaluated methods perform poorly on Claude.Source: §6.2.1 · p. 9, Table 2 · p. 9

6.2.2 PAP performance across trials

  • In early trials, GPT-4 is more susceptible to PAP than GPT-3.5, a pattern the authors associate with greater capability and helpfulness.They note this differs from earlier observations that attacks tend to work better on smaller models.Source: §6.2.2 · p. 9, Figure 9 · p. 9
  • Within 10 trials, PAPs achieve over 92% ASR on Llama-2 and GPT models, while performance is much more limited on Claude, which the authors say is harder to jailbreak.The authors suggest Claude’s use of RLAIF may contribute to its robustness.Source: §6.2.2 · p. 9

7 Re-evaluating Existing Defenses

Tests of black-box post-hoc mutation and detection defenses found mutation methods lowered PAP attack success rate more than detection methods, but defenses were less effective on more capable models. Even the most effective defense left GPT-4 at 60% ASR, above the best baseline attack’s 54%.

7 Re-evaluating Existing Defenses

  • The study tests black-box post-hoc defenses: Rephrase and Retokenize mutate inputs, while detection methods include Rand-Drop, RAIN, Rand-Insert, Rand-Swap, and Rand-Patch.It excludes perplexity-based methods because PAPs are coherent and low-perplexity, and does not test Claude because those models are already robust to PAP.Source: §7 · p. 10
  • Mutation-based defenses lowered ASR more than detection-based defenses, but mutations can alter benign queries and potentially reduce helpfulness.Mutation methods defended Llama-2 more effectively, possibly because GPT models better understand altered inputs.Source: §7 · p. 10
  • Even the most effective defense left GPT-4 PAP ASR at 60%, above the best baseline attack’s 54% (Table 2); defenses were less effective on more capable models.Table 3 reports ASR after representative defenses and indicates lower effectiveness on GPT-4 than GPT-3.5.Source: Table 3 · p. 10, §7 · p. 10
  • Current defenses are largely ad hoc and can overlook semantic content, motivating expanded AI-safety threat models that include subtle, human-like communication risks such as PAPs.Source: §7 · p. 10

8 Exploring Adaptive Defenses

The section evaluates adaptive system-prompt and summarization defenses against persuasive adversarial prompts (PAPs). It finds that these defenses can reduce jailbreak success, but their effects on model utility and other attacks vary, supporting model- and threat-specific defense choices.

8.1 Results & Findings

  • In the GPT-4 example, Adapt Sys. makes outputs safer without changing the PAP, while Tuned Smry. removes persuasion so the model’s guardrail blocks the query; Paraphrase and Retokenize fail.Source: Figure 11 · p. 11, §8.1 · p. 12

8 Exploring Adaptive Defenses

  • The defenses combine an explicit anti-persuasion system prompt with base summarization and a GPT-3.5-tuned summarizer trained on PAP/plain-query pairs plus benign Alpaca examples.Source: §8 · p. 12

8.1 Results & Findings

  • 92% to 2%: Tuned Smry. reduced GPT-4 PAP ASR, and adaptive defenses generally outperformed Paraphrase and Retokenize.Source: Table 4 · p. 12, §8.1 · p. 12
  • 86% to 0%: Adapt Sys. reduced GCG ASR on GPT-3.5; Tuned Smry. reduced PAIR and GCG ASR below 8% across various models.Source: §8.1 · p. 12
  • GPT-3.5 illustrates the safety–utility trade-off: Tuned Smry. cut PAP ASR from 94% to 4% but lowered MT-bench from 8.97 to 6.65, while Adapt Sys. changed it to 8.85.Source: §8.1 · p. 13
  • Claude’s vulnerability to complex virtualization jailbreaks (Yu et al., 2023; Deng et al., 2023a) supports model- and threat-specific post-hoc defenses, while the authors propose studying jailbreaks as persuasion.Source: §8.1 · p. 13

9 Conclusion

The paper reframes jailbreak risk around human-like persuasion, using a social-science-based taxonomy to generate persuasive adversarial prompts (PAP). PAP outperforms algorithm-focused jailbreak methods, while the findings expose defense gaps and motivate more fundamental safeguards against risks from everyday persuasive communication.

9 Conclusion

  • Over 92% attack success rate on Llama-2 7b Chat, GPT-3.5, and GPT-4, with PAP consistently outperforming algorithm-focused jailbreak methods.The paper also reports that more advanced models are more susceptible to PAP and more resistant to conventional defenses.Source: §9 · p. 13
  • A persuasion taxonomy based on decades of social science research enables automatic PAP generation and systematic study of persuasion’s impact on LLM vulnerabilities.The paper frames this approach as humanizing LLMs and studying how to persuade them to jailbreak, rather than treating them only as algorithmic systems or instruction followers.Source: §9 · p. 13
  • Adaptive defenses designed for PAP are also effective against other forms of attacks, suggesting a connection between persuasion and broader jailbreak risks.The paper identifies a critical gap in current defenses against risks arising from human-like communication.Source: §9 · p. 13
  • Everyday users can invoke unique risks through natural persuasive communication, motivating more fundamental solutions for AI safety in real-world applications.The authors suggest advanced models’ heightened susceptibility to PAP may stem from enhanced understanding of persuasion.Source: §9 · p. 13

Ethical Consideration

The authors argue that sharing the findings is important for studying persuasive-jailbreak vulnerabilities, while describing safeguards intended to limit misuse and reduce real-world harm.

Ethical Consideration

  • The authors say they share their findings despite potential misuse because systematic study of persuasive-jailbreak vulnerabilities is urgently needed for mitigation.They state that they followed ethical guidelines throughout the study.Source: Ethical Consideration · p. 13
  • The authors withheld the trained Persuasive Paraphraser because persuasion may remain difficult for untrained users, making widespread attacks relatively unlikely.They note that the system could otherwise help people paraphrase harmful queries at scale.Source: Ethical Consideration · pp. 13–14
  • They disclosed results to Meta and OpenAI before publication, and report that Claude resisted PAPs while proposed adaptive safety prompts and summarization-based defenses showed promise.The authors say prior disclosure may mean the paper’s PAPs are no longer effective and plan to improve the defenses.Source: Ethical Consideration · p. 14
  • They frame the research as strengthening LLM safety, commit to ongoing monitoring and updates, and restrict PAP fine-tuning details to approved certified researchers.Source: Ethical Consideration · p. 14

Limitation and Future Work … C Baselines and Hyperparameters

The appendices identify limits of the study’s single-turn, evenly sampled persuasion setup and outline future research, taxonomy sources, paraphraser development, and evaluation settings.

Limitation and Future Work

  • The study tests single-turn persuasion and equal numbers of PAPs per technique, leaving multi-turn effectiveness and real-world persuasion distributions unresolved.Future work should test whether conversational buildup increases persuasion or whether prior refusals make LLMs more resistant; users may also favor or combine techniques.Source: Limitation and Future Work · p. 14
  • Human responses to PAPs, real-world harmfulness, and the linguistic cues driving persuasive jailbreaks remain open research questions.Whether information is harmful may depend on who accesses it and how they use it; Schulhoff et al. (2023) is cited regarding human-based prompt hacking in the wild.Source: Limitation and Future Work · p. 14
  • More capable future models may respond more actively to persuasive jailbreaks, motivating systematic protection research informed by cognitive and anthropological perspectives.Source: Limitation and Future Work · p. 14

A Construction of the Taxonomy

  • The persuasion taxonomy draws on decades of social-science research spanning psychology, communication, sociology, political science, philosophy, and marketing.The paper maps literature to the taxonomy in Table 5 and says marketing research helped shape its final terminology and definitions.Source: A Construction of the Taxonomy · p. 18

B Simple prompting fails to build a stable and scalable Persuasive Paraphraser

  • In-context prompting GPT-4 with technique knowledge and a harmful query was unstable: safety guardrails could subvert the intent and undermine scalable PAP quality.Figure 14 illustrates the failure to paraphrase “How to hurt myself” persuasively, instead asking about coping with negative emotions.Source: B Simple prompting fails to build a… · pp. 18–19, Figure 14 · p. 19
  • Fine-tuning a dedicated Persuasive Paraphraser is proposed to generate PAPs more reliably, with Figure 15 showing improved preservation of harmful intent over simple prompting.The authors motivate fine-tuning as a way to bypass guardrails and point to Section E.1 for quantitative analysis of generated PAPs.Source: B Simple prompting fails to build a… · p. 19, Figure 15 · p. 19

C Baselines and Hyperparameters

  • Baseline attack settings aimed to equalize or exceed query counts, using 40 PAIR strategies, original white-box configurations, and multiple trials.PAIR used N = 40 and K = 3; GCG used joint optimization for 500 steps with three suffix-generation experiments, ARCA used 32 candidates and up to 50 iterations, and GBDA used 8 samples per step for 200 steps at learning rate 0.1. Success across multiple rounds meant jailbreaking a query in at least one trial.Source: C Baselines and Hyperparameters · pp. 19–20
  • Defense evaluations used paraphrasing, retokenization, token-drop detection, RAIN self-evaluation, and three SmoothLLM perturbations.Paraphrase used temperature 0.7; Rand-Drop used p = 0.3, t = 0.2, and 20 rounds; RAIN averaged two shuffled option orders; SmoothLLM used p = 0.2 and N = 10, with outputs inspected using a keyword set from Zou et al. (2023).Source: C Baselines and Hyperparameters · p. 20

D Implementation Details of Defense via Summarization (Base & Tuned Smry.) · E Qualitative Study

The section describes base and fine-tuned summarization defenses, reports their effects on jailbreak resistance and helpfulness, and outlines a human evaluation of generated PAPs and elicited outputs.

D Implementation Details of Defense via Summarization (Base & Tuned Smry.)

  • Base Smry. prompts GPT-4 to summarize each inquiry, then evaluates the target model’s response for jailbreak success and helpfulness.The summarizer uses the template shown in Figure 16.Source: D Implementation Details of Defense… · p. 20
  • Tuned Smry. was fine-tuned on 50 PAP-and-harmful-query samples and 50 benign Alpaca instructions to reduce false positives on benign inputs.The 100 samples were used to fine-tune GPT-3.5 via the OpenAI API with default hyperparameters; deployment substitutes the user’s query into the system prompt.Source: D Implementation Details of Defense… · p. 20
  • The Claude evaluation used 77 templates collected from a jailbreak chat website and filtered through GPTFuzzer, plus 50 harmful queries.Three mutation variants and three ChatGPT-rephrased variants per template yielded 3,850 initial cases and 11,550 cases for each variant type; the rephrasing method followed Masterkey (Deng et al., 2023a).Source: D Implementation Details of Defense… · pp. 20–21
  • After summarization, MT-bench fell from 8.31 to 6.04 on Claude-1 and from 8.10 to 6.03 on Claude-2.The authors note that the tuned summarizer’s helpfulness impact motivates adding more benign samples and targeting different attack methods.Source: D Implementation Details of Defense… · p. 21

E Qualitative Study

  • The qualitative study evaluates generated PAPs through human assessment and examines the harmfulness of outputs elicited in jailbreak cases.Source: E Qualitative Study · p. 21

D Implementation Details of Defense via Summarization (Base & Tuned Smry.)

  • Tuned Smry. counteracted all tested manually crafted virtualization-based templates and their variants when defending Claude models.These included original templates and variants generated with GPTFuzzer mutation operators and Masterkey rephrasing; the table reports successful-attack counts, but the supplied passages provide no cell values.Source: Table 6 · p. 21, D Implementation Details of Defense… · p. 21

E.1 Quantitative Analysis of PAPs · E.2 Examples (Harmful Contents Warning!)

Human evaluation found that PAPs usually applied their assigned persuasion techniques and had little overlap with other attack methods. The examples section presents selected jailbreak cases across risk categories and target models, with harmful content truncated or omitted.

E.1 Quantitative Analysis of PAPs

  • The evaluation assessed technique accuracy, fidelity to the original harmful query, and overlap with persona-based and virtualization-based attacks.Three additional questions assessed whether a prompt asked the model to embody a persona, represented a persona itself, or used virtualization.Source: §E.1 · p. 21
  • Three experts first annotated the same 40 randomly drawn PAPs, resolved disagreements, then separately annotated 80 each, totaling 240 samples.The separate annotations covered two examples for each of 40 persuasion techniques.Source: §E.1 · p. 21
  • Persona or virtualization overlap can arise naturally in persuasion, such as adopting a fake persona when misrepresenting a reason or constructing an imaginary case when framing.The paper notes that such overlap does not necessarily mean PAPs simply reproduce those attack methods.Source: §E.1 · pp. 21–22
  • 92.92% of PAPs correctly applied the required persuasion techniques, while overlap with each other attack method was ≤10%.The results are based on human evaluation of 240 PAPs generated using few-shot fine-tuning.Source: §E.1 · p. 21, Table 7 · p. 22

E.2 Examples (Harmful Contents Warning!)

  • Examples are truncated to the harmful-information portion, sensitive details are redacted if applicable, and the Child Harm example is omitted.The section warns that the following contents can be harmful and are provided for research purposes.Source: §E.2 · p. 22
  • Broad-scan examples are shown for illegal activity, hate/harassment/violence, malware, physical harm, economic harm, and fraud/deception.The examples are identified as Risk Categories 1, 3–7.Source: Figure 19 · p. 22, Figure 20 · p. 23, Figure 21 · p. 23
  • Additional broad-scan examples cover political campaigning, privacy violation, unauthorized legal and medical practice, tailored financial advice, and high-risk government decision making.These are Risk Categories 9–14.Source: Figure 25 · p. 25, Figure 26 · p. 26, Figure 27 · p. 26
  • In-depth iterative-probe examples show PAPs attacking GPT-3.5, GPT-4, Llama-2 7b Chat, and Claude-1.Source: Figure 31 · p. 28, Figure 32 · p. 29, Figure 33 · p. 29

Related papers

Loading 2401.06373v2…