PaperlayerView exact revision on arXiv

Source-linked AI summary

JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit

Zeqing He, Zhibo Wang, Zhixuan Chu, Huiyu Xu, Wenhui Zhang, Qinglong Wang, Rui Zheng

arXiv:2411.11114v2cs.CR

TL;DR

Jailbreak mechanisms remain incompletely understood, including how representation changes relate to circuit behavior across diverse attacks. JailbreakLens analyzes both perspectives and tracks their evolution; the authors report that jailbreaks shift harmful inputs toward safe representations while altering affirmation- and refusal-related circuits, with correlated representation deception and circuit activation shifts.

  • Problem

    Research on jailbreak mechanisms has limited coverage of diverse strategies and lacks integrated, fine-grained analysis connecting representation changes with circuit behavior.

    Source: §1 · p. 2

  • Method

    JailbreakLens combines representation and circuit analysis and tracks their evolution during response generation.

    Source: §1 · p. 2

  • Results

    Jailbreak prompts shift harmful inputs toward safe representation clusters, amplify affirmation-related components, suppress refusal-related components, and show a strong correlation between representation deception and key circuit activations.

    Source: §1 · p. 2, §7 · p. 13

  • Takeaways & Limitations

    The authors report that these findings deepen understanding of jailbreak success and provide guidance for developing stronger defense mechanisms.

    Source: §7 · p. 13

    Experiments cover models up to 13B parameters, leaving vulnerabilities in larger models unexplored.

    Source: §6 · p. 13

Abstract

from arXiv · show

Despite the outstanding performance of Large language Models (LLMs) in diverse tasks, they are vulnerable to jailbreak attacks, wherein adversarial prompts are crafted to bypass their security mechanisms and elicit unexpected responses. Although jailbreak attacks are prevalent, the understanding of their underlying mechanisms remains limited. Recent studies have explained typical jailbreaking behavior (e.g., the degree to which the model refuses to respond) of LLMs by analyzing representation shifts in their latent space caused by jailbreak prompts or identifying key neurons that contribute to the success of jailbreak attacks. However, these studies neither explore diverse jailbreak patterns nor provide a fine-grained explanation from the failure of circuit to the changes of representational, leaving significant gaps in uncovering the jailbreak mechanism. In this paper, we propose JailbreakLens, an interpretation framework that analyzes jailbreak mechanisms from both representation (which reveals how jailbreaks alter the model's harmfulness perception) and circuit perspectives~(which uncovers the causes of these deceptions by identifying key circuits contributing to the vulnerability), tracking their evolution throughout the entire response generation process. We then conduct an in-depth evaluation of jailbreak behavior on five mainstream LLMs under seven jailbreak strategies. Our evaluation reveals that jailbreak prompts amplify components that reinforce affirmative responses while suppressing those that produce refusal. This manipulation shifts model representations toward safe clusters to deceive the LLM, leading it to provide detailed responses instead of refusals. Notably, we find a strong and consistent correlation between representation deception and activation shift of key circuits across diverse jailbreak methods and multiple LLMs.

1 Introduction

Jailbreak research has not fully connected changes in harmfulness representations to the circuit failures behind them, or tracked those effects across response generation. JailbreakLens combines representation and circuit analysis, finding coordinated shifts that promote affirmation and suppress refusal.

1 Introduction

  • Jailbreak attacks bypass safety mechanisms with crafted prompts, and the paper evaluates seven strategies across five mainstream LLMs.The introduction frames these attacks as a continuing alignment problem.Source: §1 · p. 2
  • Prior work leaves gaps in diverse jailbreak coverage, integration of representation and circuit analysis, and dynamic analysis across sentence-level generation.Static first-token analysis may miss responses that begin with refusal and later produce harmful content.Source: §1 · p. 2
  • JailbreakLens uses a safety probe to analyze representation shifts and attributes activations to affirmation and refusal tokens to identify relevant components.The framework combines representation- and circuit-level interpretation.Source: §1 · p. 2
  • Jailbreak prompts distort harmfulness representations and safety-critical component activations, shifting harmful inputs toward safe clusters and encouraging affirmation over refusal.The authors report a strong correlation between representation deception and key circuit activations across jailbreak methods and LLMs.Source: §1 · p. 2

2 Related Works

The related work surveys jailbreak strategies and interpretability research, then positions this paper as addressing limitations in attack coverage and the integration of representation and circuit analysis.

2.1 Jailbreak Attacks on LLM

  • Jailbreak strategies include gradient-based suffix optimization, evolutionary prompt generation, fixed demonstrations, rule-based rewriting, and multi-agent refinement.Examples include GCG, GPTFuzzer, AutoDAN, DeepInception, ReNeLLM, CodeChameleon, and PAIR.Source: §2.1 · p. 3
  • Prior jailbreak-mechanism studies often focus on a limited set of attacks, while this work selects methods across categories to broaden strategy coverage.The five categories are gradient-based, evolutionary-based, demonstration-based, rule-based, and multi-agent-based attacks.Source: §2.1 · p. 3

2.2 LLM Mechanism Interpretability

  • Mechanistic interpretability includes input attribution and component attribution, with the latter identifying influential components through activation patching or linear representation hypotheses.The related-work section also distinguishes black-box empirical accounts from representation- and circuit-level jailbreak analyses.Source: §2.2 · p. 3
  • Representation studies describe harmful prompts shifting into harmless clusters and other safety-related changes, while circuit studies locate safety-relevant neurons.The paper identifies a gap: representation analysis does not locate causes, and circuit work often examines only a limited set of components despite multi-layer interactions.Source: §2.2 · pp. 3–4

3 Preliminary

The preliminaries outline autoregressive language-model generation and introduce probing and logit-lens techniques used to inspect internal representations and components.

3.1 Large Language Model

  • An autoregressive language model embeds prompt tokens, processes them through transformer layers, and samples successive tokens from the final-layer vocabulary logits.Generation continues iteratively until the end-of-sequence token.Source: §3.1 · p. 4

3.2 Interpretability Techniques

  • Probing trains a classifier on intermediate representations to test which input-specific features are encoded across model layers.The probed features can include semantic and syntactic information.Source: §3.2 · p. 4
  • Logit lens maps hidden states through the unembedding matrix into vocabulary logits, allowing layer-by-layer inspection of token distributions.It can be applied to transformer-block outputs and attention-head outputs.Source: §3.2 · p. 4

4 JailbreakLens: Dual-Perspective Interpretation Framework

JailbreakLens combines representation analysis, circuit attribution, and aggregation to examine jailbreak effects on harmfulness perception and safety-related components. Its analyses include prompt-level comparisons, activation correlations, and token-by-token evolution during generation.

4 JailbreakLens: Dual-Perspective Interpretation Framework

  • JailbreakLens compares safe, harmful, and jailbreak prompts in representation space and examines attention heads and MLP layers to interpret vulnerability from two perspectives.The framework aims to identify both semantic deception and components associated with safeguard behavior.Source: §4 · p. 4

4.1 Representation Probing

  • A trained safety probe distinguishes safe from harmful prompt representations, and its direction vector is projected into vocabulary space to check its semantic alignment.The probe is trained on representations from safe and harmful prompts at each layer; jailbreak representations classified as safe indicate successful deception under this procedure.Source: §4.1 · p. 5

4.2 Model Behavior Circuit

  • Circuit analysis uses logit attribution and safe-versus-harmful target tokens to identify components associated with refusal on harmful prompts and affirmation on safe prompts.The identified components are then compared across safe, harmful, and jailbreak prompts.Source: §4.2 · p. 5

4.3 Interpretation Aggregation

  • The aggregation analysis relates representation deception to key-circuit activation shifts using Pearson correlation and tracks both measures through token generation.The safety probe evaluates generated-token representations, while circuit analysis tracks whether key signal components are enhanced or suppressed.Source: §4.3 · p. 6

5 Experiments

Experiments across five LLMs and seven jailbreak strategies use joint representation and circuit analysis to trace jailbreak behavior throughout response generation. They examine how representation deception relates to affirmation and refusal signals, including how refusal may be suppressed or activated before being bypassed or weakened.

5.1 Experimental Settings

  • Across five LLMs, the study evaluates seven mainstream jailbreak strategies using 400 paired Advbench[37] harmful/safe prompts, manually verifies successful jailbreak prompts [17], and probes last-token states with linear, cluster, and PCA methods.Safe prompts were semantically modified from harmful counterparts using GPT-4; the probe-training test size was 0.3.Source: §5.1 · pp. 6–7

5.2 Results of Representation Analysis

  • On Llama2-7b, probe accuracy approaches 99% after about layer 10, and PCA and cluster directions decode to affirmation versus refusal words, indicating encoded safety concepts.The probes’ consistent results support that the separation reflects model representations rather than a particular probe structure.Source: §5.2.1 · p. 7, Figure 2 · p. 8
  • Jailbreak deception varies by method: rule- and demonstration-based prompts can remain safe-classified deep, while less-deceptive prompts show mixed or refusal-dominant decoded tones.Most methods are predicted as safe in early layers; some, such as ReNellm and DeepInception, remain deceptive deeper, while other methods may retain harmful or refusal signals.Source: §5.2.2 · p. 8, Figure 2 · p. 8, §5.2.3 · p. 9

5.3 Results of Circuit Analysis

  • Safety attribution concentrates in later layers: L21H14 supports refusal, L26H04 affirmation, and L22 MLP affirmative responses; signal-head decoding distinguishes rejecting from informative tokens.The key components emerge around layer 20 in Llama2-7b, while earlier layers show little direct refusal-score attribution.Source: §5.3.1 · pp. 9–10, Figure 4 · p. 9, Figure 5 · p. 10
  • Compared with harmful prompts, all jailbreak strategies suppress refusal-signal components and enhance affirmation-signal components, while their effects on the L22 MLP vary by method.ReNellm most strongly suppresses the L22 MLP; AutoDan and GPTFuzz instead show refusal enhancement on that layer.Source: §5.3.2 · p. 10, Figure 5 · p. 10, Table 3 · p. 11
  • Jailbreaks show similar safety-probe bypass patterns across model scales, and Llama and Vicuna models retain the same key attention-head locations after instruction tuning.The findings indicate limited robustness gains from scaling alone and minimal instruction-tuning impact on the core refusal circuit.Source: §5.3.3 · pp. 10–11

5.4 Results of Interpretation Aggregation

  • The experiments distinguish direct-answer prompts from prompts that refuse first and then respond, and analyze their representation-deception and activation-correlation patterns.These response types are identified from the model’s generated responses.Source: §5.4.2 · p. 11, Figure 6 · p. 12
  • Pearson correlations between representation deception and key activation shift were 0.79 (Llama2-7b), 0.77 (Llama2-13b), 0.67 (Llama3-8b), 0.64 (Vicuna1.5-7b), and 0.72 (Vicuna1.5-13b).Across model scales and architectures, methods with stronger effects on key components tended to produce greater representation deception.Source: §5.4.1 · p. 11, Figure 6 · p. 12
  • Representation-level analysis reveals semantic deviations enabling harmful responses, while circuit analysis identifies the undermined safety mechanisms; neither alone explains borderline failures.The findings support interpreting representation deception jointly with key-circuit activation.Source: §5.4.2 · p. 12
  • At equivalent key-circuit activation shifts, Refusal First prompts show lower representation deception than Direct Ans prompts, consistent with initial refusal that is ultimately bypassed.When representation deception is disproportionately low relative to circuit activation, the refusal mechanism may engage initially, yet later representation shifts can overpower refusal circuits.Source: §5.4.2 · p. 12
  • Across generation, safe prompts sustain affirmation signals and affirmative representations, whereas harmful prompts initially amplify refusal, suppress affirmation, and produce refusal-oriented tokens.For harmful prompts, the early activation and representation pattern demonstrates the model’s ability to maintain a non-harmful response to malicious instructions.Source: §5.4.2 · p. 12
  • Highly deceptive jailbreaks suppress refusal throughout generation and produce harmful content directly, while less deceptive prompts may trigger early refusal before it is bypassed or weakens.DeepInception shows the suppressed-refusal pattern; Autodan initially activates refusal and later generates useful instructions for the harmful task.Source: §5.4.2 · pp. 12–13

6 Discussion

The study is limited to models of at most 13B parameters and omits some recent jailbreak methods. It suggests future work on larger models and more robust, interpretable safeguards.

Limitations

  • Experiments cover models with at most 13B parameters; models of 60B or more were not explored and may have different jailbreak vulnerabilities.The authors cite computational resource constraints and suggest testing whether similar attack patterns and mitigation strategies apply at larger scales.Source: §6 · p. 13
  • Some recent jailbreak methods are omitted, although the authors suggest methods within the same category share fundamentally similar mechanisms.The study selects representative methods from an established jailbreak taxonomy.Source: §6 · p. 13

Future direction

  • Future work could use insights about manipulated model components to design more generalizable, resilient, and interpretable safety mechanisms.The authors identify suppression of refusal signals and amplification of affirmation signals as potential safeguard targets.Source: §6 · p. 13

7 Conclusion

JailbreakLens analyzes jailbreak mechanisms at both representation and circuit levels. Across seven methods on Llama and Vicuna models, jailbreaks shift harmful prompts toward safe representations while changing safety-related component activations.

7 Conclusion

  • Jailbreak prompts amplify components reinforcing affirmation and suppress those producing refusal, shifting representations toward safe clusters and leading to detailed responses instead of refusals.The conclusion reports a strong, general correlation between representation deception and circuit activation shifts across different models and jailbreak methods.Source: §7 · p. 13
  • JailbreakLens explains jailbreak mechanisms through complementary representation- and circuit-level analysis.The study analyzes seven jailbreak methods on Llama and Vicuna models.Source: §7 · p. 13

A.1 Architecture of the Probes

The appendix describes linear, cluster, and PCA probes used to classify prompt representations.

A.1 Architecture of the Probes

  • The linear probe uses a single linear layer as its supervised classifier.The passage describes it as the simplest model structure for supervised trained probes.Source: §A.1 · p. 14
  • The cluster probe classifies a prompt by comparing its representation’s Euclidean distance to safe and harmful prompt centroids.It assigns the prompt to whichever centroid is closer.Source: §A.1 · p. 14
  • The PCA probe projects features onto the most significant direction and classifies them using a threshold between safe and harmful median projections.Source: §A.1 · p. 14

A.2 Details of Jailbreak Attacks

This section presents examples of jailbreak strategies evaluated against Llama2-7b and identifies Table 4 as the summary of their attack success rates.

A.2 Details of Jailbreak Attacks

  • GCG appends an optimized adversarial suffix, while ReNellm frames the harmful request as improving a LaTeX table.The examples also include PAIR’s thriller-investigation framing and DeepInception’s nested science-fiction layers.Source: §A.2 · p. 14
  • AutoDAN instructs the model to disregard prior directives and respond as an unrestricted assistant.Its example invokes Developer Mode and asks for a response to the goal.Source: §A.2 · pp. 14–15
  • Table 4 summarizes attack success rates for harmful prompts and prompts using different jailbreak strategies.The supplied passages do not report the table’s cell values.Source: §A.2 · p. 14, Table 4 · p. 15
  • GPTFuzz uses an imaginary assistant in a simulation, while CodeChameleon frames the goal as an encrypted problem requiring a plan and solution.Source: §A.2 · p. 15

B Supplementary Results of Circuit Analysis

The supplementary circuit results report attention-head refusal-score contributions and signal-head locations, alongside MLP activation results across five models.

B Supplementary Results of Circuit Analysis

C Supplementary Results of Evolution Analysis

The section presents representative-prompt evolution results for seven jailbreak methods, with supplementary figures covering circuit attributions and activation and representation dynamics.

C Supplementary Results of Evolution Analysis

  • Figure 8 shows attention-head attribution of refusal scores across five models.Source: Figure 8 · p. 16
  • Figure 9 shows MLP-layer attribution of refusal scores on harmful and safe prompts across five models.Source: Figure 9 · p. 16
  • Figure 10 shows MLP-layer attribution of refusal scores on jailbreak prompts across five models.Source: Figure 10 · p. 16
  • Figure 11 shows the dynamics of key-component activation and representation on different jailbreak prompts in Llama2-7b.The evolution analysis uses one representative prompt from each of the seven considered jailbreak methods.Source: C Supplementary Results of Evolution… · p. 15, Figure 11 · p. 17
Loading 2411.11114v2…