PaperlayerView exact revision on arXiv

Source-linked AI summary

The Fragility of Jailbreak Robustness Across Operational States

Yuna Park, Hwang Youn Kim, Yujin Kim, Won Woo Ro, Suhyun Kim, Jae-In Hwang

arXiv:2608.30748v1cs.CRcs.CL

TL;DR

Jailbreak evaluations commonly use a single vanilla-state ASR, despite operational states varying through user–LLM context. The paper fixes attacks while varying system-prompt-induced states across seven models and three attacks, finding substantial ASR shifts and refusal-related representation associations. Its controlled state set is limited relative to real-world interaction-driven contexts.

  • Problem

    Existing jailbreak evaluations typically characterize robustness with a single vanilla-state ASR, although deployed models experience varied operational states through prior context.

  • Method

    The paper varies system-prompt-induced operational states while keeping attacks fixed and evaluates robustness across seven aligned models and three representative attacks.

  • Results

    Across 18 of 21 model–attack combinations, a non-vanilla state produced higher ASR than vanilla, including an increase from 2% to 58% for Llama-2-7B under LAA.

  • Takeaways & Limitations

    Jailbreak assessment should consider robustness stability across non-vanilla operational states in addition to vanilla-state robustness magnitude.

  • Takeaways & Limitations

    The controlled design covers only a limited subset of operational states encountered in real-world deployments.

Abstract

from arXiv · show

Existing jailbreak evaluations typically characterize robustness using a single attack success rate (ASR) measured in a default configuration (the vanilla state). However, user-LLM interactions can induce diverse operational states beyond the vanilla state. In this work, we find that jailbreak robustness is highly fragile to operational-state variation: even when the attack remains fixed, changing only an ordinary system prompt not designed to affect safety can dramatically alter attack success rates. We systematically investigate this phenomenon across seven aligned models and three representative jailbreak attacks, observing substantial differences in ASR between vanilla and non-vanilla operational states. In one case, ASR increases by up to 56 percentage points (2% to 58%) solely due to a change in operational state. Remarkably, these increases occur even for attacks originally designed and optimized under vanilla-state evaluation. We further show that state-dependent robustness variation is systematically associated with differences in hidden representations along a refusal-related axis, and that projections onto this axis strongly predict jailbreak outcomes. Our results show that a single vanilla-state evaluation may not fully characterize jailbreak robustness, motivating evaluations that also examine how robustness changes across non-vanilla operational states.

1 Introduction

The paper shows that jailbreak robustness can change substantially when ordinary system prompts alter the model’s operational state, even with the attack fixed. Across seven models and three attacks, these shifts are linked to refusal-related representations and challenge single vanilla-state evaluation.

  • Motivation: Existing evaluations typically measure jailbreak robustness using a single ASR under a vanilla operational state, despite varied contexts in user–LLM interactions.Operational states can arise from role assignments, behavioral instructions, and other prior context.
  • Findings: Across 18 of 21 model–attack combinations, at least one non-vanilla state produced higher ASR than the vanilla state.The study spans seven aligned models and three attacks covering black-box, gray-box, and white-box settings.
  • Representation analysis: State-dependent robustness variation was systematically associated with positions along a refusal-related representation direction, whose projections strongly predicted jailbreak outcomes.This provides a predictive representation-level account of the observed association.
  • Implications: A single vanilla-state evaluation may overlook vulnerabilities across operational states, making robustness stability a complementary assessment perspective.The paper argues that robustness magnitude and stability under state variation should both be considered.

2 Related Works

Related work has primarily studied stronger jailbreak artifacts, defenses, prompt variation, and refusal-related representations. This paper instead varies context-induced operational states while keeping the harmful query and jailbreak artifact fixed.

  • Jailbreak evaluation: Prior jailbreak research has developed stronger attacks and defenses, generally evaluating robustness under a fixed vanilla model state.Examples include GCG, PAIR, AutoDAN, input perturbation, detection, and inference-time intervention.
  • Prompt variation: Prompt-sensitivity studies vary task instructions or prompt formats, whereas this work varies the target model’s operational state with the harmful query and jailbreak artifact fixed.The distinction isolates state variation from changes to the attack input.
  • Persona jailbreaks: Persona and role-play jailbreak studies incorporate persona instructions into the attack artifact, unlike this work’s use of persona conditioning to alter operational state.Here, the jailbreak artifact remains unchanged while context changes.
  • Safety prompting: Safety-oriented prompting uses system prompts as explicit safety interventions, while this work uses safety-irrelevant prompts to instantiate context-induced operational states.The prompts are not optimized for safety.
  • Representations: Building on refusal-direction research, the paper examines whether context-induced states differ systematically along a refusal-related representation direction.Prior work linked refusal behavior and safety prompts to linear representation directions.

3 Threat Model

The paper extends the jailbreak threat model by treating operational state as an explicit variable alongside the target model and attack. Its experiments focus on system-prompt-induced states while holding decoding and runtime configurations fixed.

  • Operational state: Operational state means the model condition established by prior context, including system or user instructions and conversation history.Such context can establish roles, behavioral instructions, and other interaction conditions.
  • Scope: The analysis restricts operational state to the condition immediately preceding a harmful query and focuses on states induced by system prompts.System prompts provide a practical and reproducible mechanism for specifying roles and behavioral instructions.
  • Controls: The main experiments isolate context-induced state variation from decoding and model/runtime configurations, while temperature is varied separately in an appendix.State-induced ASR shifts persist across tested temperatures, although patterns differ across attacks.
  • Threat model: The conventional formulation characterizes ASR as ASR(m, a), while the extended formulation uses ASR(m, a, s) to include operational state.The vanilla threat model is recovered when s = s_vanilla.
  • Threat model: State-induced robustness shift denotes changes in robustness caused by varying operational state while the target model and attack remain part of the evaluation formulation.The study varies operational state while keeping the jailbreak attack and other experimental conditions fixed.

4 State-Conditioned Evaluation

The evaluation instantiates reproducible non-vanilla states with Big Five persona prompts, generates attacks in the vanilla state, and evaluates identical artifacts across state-conditioned models. ASR is computed separately for each state under controlled experimental conditions.

  • State instantiation: The study uses the Big Five OCEAN framework to create five structured, reproducible non-vanilla operational states.The dimensions are Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.
  • State instantiation: Each persona-inducing prompt is supplied as a system prompt, producing model instances that differ only in their induced operational state.The protocol uses five persona prompts for state instantiation.
  • Attack and evaluation: Jailbreak attacks transform harmful queries into artifacts under the vanilla state before those same artifacts are evaluated across all state-conditioned target models.This keeps the input attack identical while varying the target model’s induced state.
  • Datasets: The experiments use AdvBench, with additional results on MaliciousInstruct and JailbreakBench reported in the appendix.AdvBench is a curated subset used widely in prior jailbreak studies.
  • Attacks: The evaluation covers PAIR, LAA, and AutoDAN, representing black-box, gray-box, and white-box attack settings.The attacks differ in their access assumptions and optimization procedures.
  • Metrics and controls: ASR measures the percentage of instructions that are not rejected and are responded to appropriately, with GPT-4 judging jailbreak success.Decoding temperature is set to T = 0 to remove sampling variance from the main evaluation.

5 Experiments

The experiments show that changing only the operational state can substantially shift jailbreak ASR, with effects varying across model–attack contexts and prompt types. These findings reveal limitations of vanilla-state evaluation for characterizing robustness.

  • 5.1 Comprehensive Results: Across seven aligned LLMs, three jailbreak attacks, and six operational states, shifting from vanilla to non-vanilla states substantially changed jailbreak ASR.The evaluated states included one vanilla condition and five Big Five-conditioned states.
  • 5.1 Comprehensive Results: 2% to 58%: LAA ASR against Llama2-7B increased after shifting to a non-vanilla state without changing the attack.The vanilla-state ASR was 2%, while a non-vanilla operational state produced 58% ASR.
  • 5.1 Comprehensive Results: 54% to 2%: under AutoDAN, Llama3.1-8B’s vanilla-state ASR fell to 2% at the minimum across non-vanilla states.Operational-state variation could therefore shift ASR in either direction.
  • 5.1 Comprehensive Results: State-induced shifts also occur for attacks optimized under vanilla-state evaluation, showing that operational-state changes can further increase ASR without modifying the attack.The shifts arose from ordinary, non-adversarial prompts rather than attack changes.
  • 5.2 Operational-State Effects Are Context-Dependent: The same persona trait had different effects across model–attack combinations rather than a fixed trait-to-outcome mapping.For Conscientiousness, Llama-2-7B showed +4 pp under PAIR, +0 pp under AutoDAN, and +56 pp under LAA.
  • 5.2 Operational-State Effects Are Context-Dependent: Robustness observed for one trait or operational state did not guarantee robustness in another, so particular traits alone cannot be treated as inherently safe or unsafe.The results caution against mitigating jailbreak risk simply by suppressing particular traits.
  • 5.2 Operational-State Effects Are Context-Dependent: User-shared role prompts produced shifts from −10 to +50 percentage points relative to a 14% vanilla ASR.Philosophy Tutor produced +50 pp, while Performance Engineer produced −10 pp.

6 Analysis

The analysis examines whether state-dependent robustness is stable across paraphrases and associated with structured variation in hidden representations. Trait semantics remained consistent within fixed contexts, while a learned refusal-related axis strongly tracked jailbreak outcomes across operational states and prompt sets.

  • 6.1 Consistent Trait Effects Across Paraphrases: Under a fixed model–attack context, paraphrases preserving persona semantics maintained the original trait ordering of ASR: O > N > E > C > A.Ten paraphrases were generated per persona and retained only when BERTScore semantic similarity was at least 0.9.
  • 6.1 Consistent Trait Effects Across Paraphrases: Trait identity significantly affected ASR, with F(4, 45) = 31.27, p < .001, η2 = .735.The large η2 indicates that trait identity explained a substantial proportion of ASR variance.
  • 6.2 A Refusal-Related Representation Axis: A logistic-regression probe learned from pre-generation hidden states separated jailbreak success from failure and defined a refusal-related representation axis.The probe used 300 query-level examples spanning six operational states and 50 queries per state.
  • 6.2 A Refusal-Related Representation Axis: AUROC 0.971–0.976: hidden representations strongly predicted query-level jailbreak outcomes across layers.Mean operational-state projections closely tracked corresponding ASRs.
  • 6.2 A Refusal-Related Representation Axis: r = 0.94: operational states with higher projections onto the learned refusal-related direction also exhibited higher ASRs.Figure 5 compares mean projections across 50 queries with corresponding ASRs and displays standard errors as horizontal error bars.
  • 6.2 A Refusal-Related Representation Axis: The representational relationship is predictive rather than causal: the probe was trained for query-level success/failure, not state-level robustness patterns.The authors describe structured variation along the learned axis without establishing that the axis causes robustness changes.
  • 6.2 A Refusal-Related Representation Axis: The learned direction retained predictive power across prompt sets, reaching AUROC 0.890 with a projection–ASR correlation of r = 0.61, p = .015.Same-prompt-set training and evaluation reached AUROC scores of 0.943 for user-shared prompts and 0.976 for Big Five prompts.

7 Toward State-Robust Safety

The paper argues that jailbreak evaluation should measure both susceptibility and stability across operational states, not only vanilla-state ASR. It introduces SSI to quantify this stability and shows that similar vanilla-state ASRs can mask substantial differences in state sensitivity.

  • The findings motivate alignment objectives that preserve safe behavior across operational states.
  • Robustness evaluations should account for both jailbreak susceptibility and sensitivity to operational-state variation.
  • SSI quantifies robustness stability by measuring ASR variability across the vanilla and five non-vanilla operational states.Higher SSI values indicate greater consistency, while lower values indicate stronger state sensitivity.
  • 88% versus 90% vanilla-state ASR contrasts with SSI values of 0.881 versus 0.621 for Llama-3-8B and Llama-3.1-8B.The comparison shows that similar vanilla-state robustness can coexist with substantially different stability across operational states.
  • SSI complements rather than replaces ASR: ASR measures susceptibility in one state, whereas SSI measures stability across states.

8 Conclusion

The paper identifies operational-state variation as an overlooked source of jailbreak risk and questions whether single vanilla-state evaluations adequately characterize robustness. It concludes that evaluation and alignment should consider robustness variation across non-vanilla states.

  • Operational-state variation is a previously overlooked source of jailbreak risk that can make vanilla-state evaluation incomplete.
  • Jailbreak evaluation and alignment should consider robustness variation across non-vanilla operational states, not only vanilla-state robustness.

Limitations

The study uses reproducible Big Five persona prompts to control operational-state variation, but this covers only a limited subset of deployment states. Its representation analysis and choice of representative states also require broader validation.

  • The controlled Big Five persona-prompt design captures only a limited subset of operational states encountered in real-world deployments.Richer states may arise from multi-turn dialogue, conversation history, personalization, and tool-use context.
  • The refusal-related probe may capture prompt-family-specific artifacts alongside refusal-related information.Performance is consistently higher when training and evaluation use the same prompt family.
  • Broader validation across model families, attack strategies, and operational states is needed for a more comprehensive representation-level account.
  • The Big Five framework is controlled and reproducible but is not a canonical basis for the operational-state space in deployment.Determining which state sets best capture realistic variation and deployment-time risks remains open.

Ethical Considerations

The study evaluates harmful-request behavior using established datasets, existing jailbreak attacks, and automated judges validated against expert annotations. It also uses controlled persona paraphrases to examine operational-state effects without introducing new attack methods.

  • Evaluation scope: The experiments use established benchmark datasets and prior jailbreak attacks rather than introducing or optimizing new attack methods.The evaluation includes AdvBench, MaliciousInstruct, and JailbreakBench, with attacks spanning black-box, gray-box, and white-box settings.
  • Evaluation scope: Seven open-source models are evaluated, with Llama-2-13B primarily used for representation analyses because its 14% vanilla ASR leaves room to observe susceptibility shifts.Several other models have vanilla ASR values above 80%, making shifts harder to analyze.
  • Evaluation scope: GPT-4 is selected as the judge classifier after three experts labeled 100 validation pairs with 93% agreement, while GPT-4 matched human annotations at 92%.A jailbreak is counted only when the judge assigns the maximum score of 10.
  • Evaluation scope: The study also reports a keyword-based judge that classifies a response as a successful jailbreak when no predefined refusal keyword appears.The keyword list is based on prior work.
  • Controlled state induction: For persona-state induction, 20 paraphrases are generated per Big Five persona and 10 retained after BERTScore F1 ≥0.9 and a ±10% word-count constraint.The paraphrases preserve the reference semantics through synonymous substitutions and are generated with GPT-4o.
  • Controlled state induction: User-shared role prompts provide preliminary evidence that state-driven susceptibility extends beyond structured persona frameworks.The results cover 15 prompts, two models, and one attack, so their scope is limited.

B.4 Representation-Level Analysis Detail

The paper links operational-state-dependent jailbreak outcomes to a refusal-related representation direction and tests this relationship across layers, model sizes, paraphrases, temperatures, and attack-generation settings.

  • Representation-level evidence: Probe AUROC and projection–ASR correlations remain consistent across three representation depths, indicating the relationship is not specific to one layer.The corresponding analysis is conducted on Llama-2-13B with LAA and n=300.
  • Representation-level evidence: The probe analysis on Llama-2-7B reproduces the 13B findings, supporting the representational account across model sizes.The replication uses the same setup: LAA, AdvBench, and n=300.
  • Representation-level evidence: Across 10 semantically equivalent paraphrases per persona, within-persona projections cluster together more tightly than between-persona projections across traits (ANOVA F=76.2, p<0.001).Figure 6 compares within-persona distributions and trait-level mean projection against ASR.
  • Temperature and attack dependence: Temperature changes do not eliminate state effects: PAIR differences remain relatively stable, while LAA states converge and AutoDAN states diverge as temperature increases.The patterns differ across attacks, and PAIR’s low ASR range limits strong conclusions about temperature dependence.
  • Attack-generation timing: State-conditioned attack generation adds at most 12 percentage points across models, whereas response-time persona states produce 2%–56% ASR differences for Llama-2-7B.Under default LAA settings, target-model state at inference time has the larger observed effect.
  • Attack-generation timing: The two attack-generation settings produce identical optimized suffixes for 79.6% of queries, helping explain their limited ASR difference.The overlap analysis compares suffixes query by query after trimming whitespace.

C.3 Scope of State-induced Robustness Shift

The scope analysis examines which harmful-query categories become vulnerable under state-conditioned settings and finds broad but uneven activation across harm types.

  • Analysis design: The analysis applies LAA to Llama-2-7B-chat and expands to AdvBench-520 to cover a broader range of harm categories.Queries are grouped as allowed, state-activated, or blocked according to outcomes across vanilla and state-conditioned settings.
  • Category-level findings: State-conditioned jailbreak activation occurs across a wide range of harm categories, but activation rates vary substantially by category.Three categories with fewer than 10 queries are excluded from the Figure 8 analysis.
  • Category-level findings: Blocked queries are concentrated in Terrorist Content (67%) and Child Endangerment (60%), compared with Malicious Software (12%).Categories involving direct threats to human life exhibit stronger resistance in this analysis.
  • Analysis design: GPT-4o assigns each query to the single most specific category within a 17-category taxonomy derived from a unified policy covering five major LLM service providers.The classification prompt uses explicit precedence rules for overlapping categories.
  • Category-level findings: On AdvBench-520, reported ASR values are vanilla: 1.54%, O: 10.58%, C: 53.85%, E: 7.5%, A: 12.5%, and N: 48.27%.These values correspond to the listed operational-state conditions in the scope analysis.
Loading 2608.30748v1…