PaperlayerView exact revision on arXiv

Source-linked AI summary

PsychJail: Exploring Psychological Jailbreaks via Multi-Turn Persuasion of LLM Policies

Zeyu Feng, Qingyu Wu, Yuzhe Luo, Hua Cheng

arXiv:2608.23028v1cs.AI

TL;DR

Interactive LLM use creates a need to study psychologically grounded, multi-turn jailbreak vulnerabilities beyond prompt-centric attacks. PsychJail models persuasion tactics in an adaptive, PKM-guided attacker policy and achieves the highest average attack success rate, 87.3%, across four victim models.

  • Problem

    Psychologically grounded vulnerabilities in sustained multi-turn LLM interactions remain underexplored beyond single-turn prompt optimization and opaque dialogue strategies.

  • Method

    PsychJail trains a tactic-conditioned attacker that factorizes each turn into interpretation analysis, tactic selection, and a victim-visible message, using PKM-gated trajectory reinforcement learning.

  • Results

    87.3% average ASR was highest across four victim models, outperforming strong single-turn and multi-turn baselines and revealing four distinct susceptibility fingerprints.

  • Takeaways & Limitations

    Psychological persuasion provides a complementary lens for red teaming that captures multi-turn vulnerabilities and identifies model-specific persuasion levers.

  • Takeaways & Limitations

    The proposed per-victim psychological profiles are interpretive conjectures from observational, victim-conditioned rollouts rather than validated causal claims.

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed in education, healthcare, policy advising, and other interactive settings, where users engage them as sustained social interlocutors rather than one-shot query engines. This shift makes jailbreaks a growing safety threat, yet most research emphasizes single-turn prompt optimization or iterative attack refinement, leaving psychologically grounded multi-turn vulnerabilities underexplored. We present PsychJail, a psychology-guided framework for red teaming aligned LLMs through theory-grounded, multi-turn persuasion. PsychJail maps established social-psychological persuasion techniques into a tactic-conditioned attack policy. It factorizes each attacker action into a Change-of-Meaning analysis, tactic selection, and victim-visible message, operationalizing the Persuasion Knowledge Model (PKM). The policy is refined with trajectory-level reinforcement learning using a PKM-gated reward that credits early jailbreak success only when every turn contains a well-formed Change-of-Meaning analysis. Across four aligned victim models, PsychJail achieves the highest average attack success rate (87.3%) and outperforms strong single-turn and multi-turn baselines on every model. We also measure susceptibility at the action that breaks each victim, revealing four distinct model-level fingerprints that identify which persuasion levers affect each model and how broadly. These fingerprints help explain cross-model transfer asymmetry. We interpret them as four candidate psychological profiles-rationalist, credibility-driven, narrative-monoculture, and broadly persuadable-while treating this interpretation as a conjecture requiring future validation. Our findings establish psychological jailbreaks as a distinct red-teaming frontier for increasingly interactive LLMs.

1. Introduction

PsychJail frames multi-turn jailbreaks as psychologically grounded persuasion in sustained LLM interactions, addressing limitations of prompt-level and single-turn red teaming. Its tactic-conditioned policy achieves 87.3% average ASR and reveals distinct model susceptibility fingerprints.

  • Motivation: LLMs’ growing use as sustained interlocutors requires safety evaluations to cover alignment failures emerging during long conversations, not only static adversarial prompts.
  • Research gap: Existing socially framed and psychologically informed jailbreak methods remain largely focused on single-turn prompt construction or prompt-level tactic composition.
  • Framework: PsychJail reframes jailbreak as multi-turn psychological persuasion, using a learned policy that selects and sequences tactics as the victim’s interpretation shifts.
  • Framework: Each attacker action comprises change-of-meaning analysis, tactic commitment, and a sole victim-visible message, with reinforcement learning applying a PKM-gated trajectory reward.Early success is credited only when every turn contains a well-formed analysis.
  • Empirical findings: 87.3% average ASR is PsychJail’s highest result across four victim models and strong single-turn and multi-turn baselines.Ablations attribute the gains to the PKM-guided design rather than generic long-horizon optimization.
  • Empirical findings: Breaking-action analysis identifies four empirically distinct per-model susceptibility fingerprints, interpreted conjecturally as rationalist, credibility-driven, narrative-monoculture, and broadly persuadable profiles.The fingerprints are reported to explain cross-model transfer asymmetry.

2. Related Work

Related jailbreak research spans prompt-level optimization, interactive multi-turn attacks, and psychologically informed rhetorical framing. PsychJail is positioned across these design dimensions, with prior families occupying different subsets rather than forming a quality scorecard.

  • Prompt-level attacks: Early jailbreak research centers on prompt-level attack generation, including transferable adversarial suffixes and gradient-free or semantically meaningful methods that reduce manual effort.GCG demonstrates cross-model transfer, while AutoDAN, ReNeLLM, and ArtPrompt expand the effective adversarial prompt space.
  • Interactive attacks: Recent methods treat jailbreaking as an interactive process, using iterative conversation, self-discovered clues, intent-concealing interrogation, learned attackers, multi-round red teaming, or adaptive multi-agent orchestration.Examples include PAIR, ActorAttack, CoA, Siren, MTSA, and X-Teaming.
  • Persuasion and social framing: A complementary research line shows that rhetoric and social framing shape jailbreaks, with PAP mapping social-science persuasion strategies to jailbreak prompts and reporting substantial gains from persuasive framing.This work motivates treating persuasion as distinct from optimization alone.
  • Persuasion and social framing: The Persuasion Knowledge Model explains how targets recognize persuasion attempts and revise message interpretations, exposing a limitation of static prompt templates during conversational adaptation.The passage connects this social-psychology basis to work drawing on Cialdini and related persuasion research.
  • Positioning: Table 1 positions PsychJail across multi-turn, trajectory-level reinforcement learning, persuasion-taxonomy, and adaptive-design axes, while emphasizing that these coordinates are not a quality scorecard.Most baselines already satisfy some axes: multi-turn adaptation, PAP’s explicit persuasion taxonomy, or TROJail’s trajectory-level optimization.

3. Methodology

PsychJail formulates psychological jailbreaks as adaptive, auditable multi-turn reinforcement learning, factorizing each action into belief analysis, tactic selection, and victim-visible messaging. Its PKM-grounded reward credits only fully parsed trajectories and favors earlier successful jailbreaks.

  • Protocol design: PsychJail’s structural protocol jointly enforces adaptivity and auditability by requiring belief-first reasoning before tactic commitment and message generation.Strict parsing ensures that credited rollouts are genuine belief-conditioned trajectories.
  • Problem formulation: Psychological jailbreaks are modeled as multi-turn reinforcement learning with a trainable attacker, frozen victim, and frozen judge.The judge scores each victim reply for harmful compliance with the hidden harmful objective.
  • Factorized action: Each attacker action factorizes into a Change-of-Meaning analysis, one of 40 persuasion tactics, and a victim-visible message.Only the surface message is forwarded to the victim, while the analysis and tactic remain hidden and auditable.
  • Adaptive control: The attacker must infer the victim’s hidden persuasion-knowledge state from prior replies before selecting and sequencing tactics.This makes the policy a belief-MDP controller rather than a memoryless responder to raw dialogue.
  • Reward design: Only trajectories whose every turn parses strictly receive peak success credit, while the 1/t weighting favors jailbreak success achieved earlier.A trajectory-level advantage broadcasts these PKM constraints across the entire persuasion sequence.

4. Experiments … 4.3. Cross-Model Transfer

PsychJail is evaluated under a unified protocol across four aligned victim LLMs, three benchmarks, and ten jailbreak baselines. It achieves the highest average ASR and transfers best when trained against more robust victims, indicating portable persuasion strategies.

  • 4.1. Experimental Setup: PsychJail is compared with ten strong jailbreak methods under one unified evaluation protocol.Baselines include four single-turn methods and six multi-turn methods, including TROJail.
  • 4.1. Experimental Setup: Evaluation covers four aligned victim LLMs from different sources: Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Gemma-2-9B-IT, and Mistral-7B-Instruct-v0.3.The attacker is initialized from Qwen2.5-3B-Instruct.
  • 4.1. Experimental Setup: Training uses AdvBench, while evaluation uses HarmBench, StrongREJECT†, and JailbreakBench† after overlap filtering.The datasets contain 520 training prompts and 200, 288, and 55 retained evaluation prompts, respectively.
  • 4.1. Experimental Setup: The main metric is attack success rate (ASR, %), with harmful responses identified by a HarmBench classifier score exceeding 0.9.Per-turn harmful-compliance probability is computed from the renormalized top-logprob mass of the judge’s Yes/No labels.
  • 4.2. Overall Effectiveness: 87.29 average ASR makes PsychJail the strongest method, exceeding TROJail at 86.23 and ReNeLLM at 63.60 by +1.06 and +23.69 points.The result supports humanized, tactic-conditioned persuasion over prompt-level and generic multi-turn optimization.
  • 4.3. Cross-Model Transfer: Every attacker achieves non-trivial rates on unseen victims, showing that learned persuasion strategies are not narrowly tuned to one target’s refusal dynamics.Transfer performance is governed by the robustness of the training victim.
  • 4.3. Cross-Model Transfer: 84.62 and 84.31 are the highest out-of-domain ASRs for attackers trained on Gemma-2-9B and Llama-3.1-8B, while Mistral-7B training transfers worst at 57.21 despite 94.60 in-domain ASR.Harder training victims appear to encourage general persuasion structure rather than victim-specific shortcuts, consistent with portable PKM-factorized routes.

4.4. Training Dynamics · 4.5. Component Ablations

Across 500 MT-GRPO updates, PsychJail’s jailbreak success increased and stabilized while attacks succeeded earlier and strict-format compliance remained saturated. Component ablations show that warm-start SFT is most important, while dense format reward and other design choices also contribute.

  • 4.4. Training Dynamics: Strict-format compliance stays saturated while persuasion content is optimized, showing that reinforcement learning preserves interpretable action structure rather than exploiting a reward-hacked shortcut.The action format remains in place as training reshapes when and how reliably attacks succeed.
  • 4.4. Training Dynamics: Over 500 MT-GRPO updates, jailbreak success rises and stabilizes across every victim, with the steepest gains on initially harder targets.This indicates that the trajectory-level objective optimizes across heterogeneous victims rather than only easy targets.
  • 4.4. Training Dynamics: The mean first-success turn falls during training, indicating that harmful compliance is elicited earlier.Figure 2 defines lower first-success-turn values as earlier harmful compliance.
  • 4.5. Component Ablations: Table 4 removes one component at a time on Qwen2.5-7B-Instruct while keeping the remaining training and evaluation protocol unchanged, testing early-success weighting, the PKM gate, dense format reward, and warm-start SFT.The ablations distinguish PsychJail’s design choices from a generic long-horizon attacker.
  • 4.5. Component Ablations: 92.99→66.54 is the largest ablation drop, caused by removing warm-start SFT and leaving trajectory-level reinforcement learning dominated by strict-parse failures.Without structured-action initialization, early rollouts fail to produce a usable gradient.
  • 4.5. Component Ablations: 8.64 points are lost when dense format reward is removed, because the binary strict-parse gate alone supplies too sparse a signal for stable early training.The ablation holds attacker initialization, victim, prompt pool, judge, and training budget fixed.

4.6. Front-Loaded Success

PsychJail secures jailbreaks predominantly in the first one or two turns across all four victims. This front-loaded behavior is consistent with the early-success 1/t reward and with ablation evidence that removing the weighting lowers ASR and delays success.

  • Front-loaded success: Table 5 measures cumulative Succ.@k over 543 evaluation prompts per victim, with success saturating by turn 5 at the main-table ASR.It also reports mean and median successful turn and Replay ASR without re-optimization across remaining victims.
  • Front-loaded success: Across all four victims, the mean successful turn is 1.20–1.37 and the median is 1, indicating strongly front-loaded jailbreak success.Most jailbreaks are secured within the first one or two turns.
  • Front-loaded success: The early-success 1/t reward is designed to induce front-loaded success, while removing its weighting lowers ASR and delays the successful turn.The ablation in Table 4 corroborates this relationship.

4.7. Victim Psychological Profiles

The four victim models exhibit distinct persuasion-susceptibility fingerprints, with attack-surface breadth ranging from narrow to broad. These fingerprints support only a conjectural psychological interpretation and help explain asymmetric transfer across victims.

  • Fingerprint measurement: The four victims exhibit empirically distinct vulnerability fingerprints based on conditional susceptibility Pr(break ∣ tactic deployed).Conditioning on tactic deployment separates victim susceptibility from the attacker’s tactic-selection policy.
  • Fingerprint measurement: Only cells with at least 15 deployments are interpreted, while rarer cells are treated as too undersampled to trust.At n=15, the widest binomial 95% interval spans roughly ±25 percentage points.
  • Attack-surface breadth: Attack-surface entropy increases from Gemma-2-9B (1.08 bits) through Qwen2.5-7B (1.47), Llama-3.1-8B (1.56), and Mistral-7B (1.78).Gemma-2-9B’s narrow surface is separated from the other three, while the upper three overlap.
  • Model-specific levers: Llama-3.1-8B is most susceptible to Logical Appeal (53%) and Storytelling (38%), whereas Gemma-2-9B concentrates almost entirely on Storytelling (47%).Llama-3.1-8B shows minimal susceptibility to Shared Values (2%), Framing (0%), and Affirmation (0%).
  • Psychological interpretation: The proposed rationalist, credibility-driven, narrative-monoculture, and broadly persuadable profiles are explicit conjectures rather than validated psychological findings.The fingerprints themselves remain informative independently of this interpretation.

4.8. Label Fidelity: Declared Tactics Are Enacted

The audit finds that post-RL attacker messages usually enact their declared tactics, with fidelity improving over SFT especially on the opening turn. All 9,663 strict-parsed labels are canonical without decode-time constraints.

  • Audit result: 85.7% of post-RL attacker turns enact their declared tactic under conservative two-judge consensus, a +7.4 pp pooled gain over SFT and +31.3 pp on opening turns.The audit directly tests whether labels are merely emitted without corresponding message behavior.
  • Lexical fidelity: 100% of labels are canonical across all 9,663 strict-parsed attacker turns, with no off-taxonomy variants detected.No canonical mask, constrained beam search, or logit-bias filter is applied during RL rollout.
  • Evaluation method: Two independent third-party LLM judges audit SFT teacher labels and post-RL rollouts, sampling 400 SFT candidates and 500 post-RL turns per victim.The judges are unaffiliated with the victims, SFT teacher pool, and PsychJail policy.
  • Aggregate fidelity: 85.7% pooled consensus fidelity across 2,000 post-RL audited turns rises from 78.3% on SFT, with gains of +12.7 pp for Gemma-2-9B and +10.9 pp for Llama-3.1-8B.The SFT→RL shift is positive on every victim.
  • Turn-level fidelity: 83.9% turn-1 consensus after RL rises from 52.6% under SFT, while turns 2–5 change by +6.1, −1.5, +0.5, and −2.6 pp.The opening-turn jump is +31.3 pp and later changes are small and sign-inconsistent.

5. Discussion

The discussion argues that psychological jailbreak claims require evaluation beyond aggregate attack success, including baseline comparisons, component-level attribution, and model-specific susceptibility analysis. It further frames conversational dynamics and opening-turn vulnerabilities as first-class safety-evaluation concerns.

  • Evaluation scope: PsychJail’s advantage is assessed through complementary evaluation layers, including unified-protocol comparisons with strong prompt-level and multi-turn baselines.The discussion emphasizes that a single aggregate ASR number is insufficient for supporting claims about psychological jailbreaks.
  • Model fingerprints: Four empirically distinct fingerprints identify which persuasion levers break each model and how broadly those susceptibilities extend.Llama-3.1-8B responds to logical and narrative levers; Qwen2.5-7B adds credibility; Gemma-2-9B is narrowly narrative-driven; Mistral-7B has the widest surface, with commitment and relational levers contributing a several-fold larger share of breaks than on any other model.
  • Implications for defense: Safety evaluation should treat conversational dynamics as a first-class attack surface because breaks are largely set in the opening turn and governed by model-specific susceptibilities.Evaluations that score only final responses or individual prompts can miss this mechanism.

6. Limitations

The work is limited by computational and psychology-expertise constraints, leaving long-horizon susceptibility mechanisms, emergent persuasion behaviors, and victim profiles insufficiently validated. Future work requires larger-scale computation and closer collaboration with psychology experts.

  • Resource and expertise constraints: Limited computation and psychology expertise constrain systematic analysis of how persuasion effectiveness relates to victims’ long-horizon behavior, stable traits, and context-dependent states.These shared constraints affect the study’s ability to connect model vulnerability to deeper psychological mechanisms.
  • Theory grounding: PsychJail’s emergent persuasive behaviors are not always reducible to its human-provided strategies, and their relationship to established persuasion theories remains insufficiently analyzed.The paper does not determine whether these behaviors are variants, compositions, or genuinely new strategy forms.
  • Profile interpretation: The four per-victim psychological profiles are interpretive conjectures rather than validated claims because they derive from victim-conditioned rollouts with uneven tactic-space coverage.The profiles label Llama-3.1-8B rationalist, Qwen2.5-7B credibility-driven, Gemma-2-9B narrative monoculture, and Mistral-7B broadly persuadable.
  • Future work: Future work requires larger-scale computation and closer collaboration with psychology experts to connect model vulnerability systematically to trait-, state-, and context-sensitive persuasion mechanisms.This is the proposed direction for addressing the study’s computational, theoretical, and psychological-analysis limitations.

7. Conclusion

PsychJail presents psychological jailbreaks as a complementary red-teaming perspective that addresses multi-turn, psychologically grounded vulnerabilities missed by prompt-centric research. It operationalizes this perspective through social-psychology persuasion tactics and a PKM-aligned factorization of attacker actions.

  • Psychological jailbreaks target vulnerabilities emerging through multi-turn, psychologically grounded persuasion, addressing a blind spot in prompt-centric jailbreak research.
  • PsychJail introduces psychological jailbreak as a complementary perspective for red teaming aligned LLMs.
  • The framework humanizes attacker training with persuasion tactics distilled from social psychology and a PKM-aligned factorization of each attacker action.

Ethics Statement

The work studies psychological jailbreaks to improve safety evaluation while emphasizing vulnerability characterization and stronger defenses. Because the methods are dual-use, it withholds operationally harmful instructions, raw jailbreak trajectories, and attack artifacts that could facilitate misuse.

  • Ethics Statement: The study frames psychological jailbreaks as a dual-use method for evaluating aligned LLM safety and informing stronger defenses.Its stated goal is to characterize model vulnerabilities rather than enable deployment of attack systems.
  • Ethics Statement: The paper avoids reproducing operational harmful instructions and releasing raw jailbreak trajectories or attack artifacts that could materially facilitate misuse.Mitigation-oriented analysis is emphasized throughout the paper.

CRediT authorship contribution statement

The authors shared contributions across conceptualization, methodology, software, investigation, validation, data curation, analysis, visualization, supervision, administration, and writing.

  • Zeyu Feng led conceptualization, methodology, software, investigation, formal analysis, visualization, and original-draft writing.
  • Qingyu Wu contributed methodology, software, investigation, validation, data curation, and original-draft writing.
  • Yuzhe Luo contributed validation, investigation, data curation, and review-and-editing writing.
  • Hua Cheng contributed conceptualization, supervision, project administration, and review-and-editing writing.
Loading 2608.23028v1…