PaperlayerView exact revision on arXiv

Source-linked AI summary

AnchorRep: Defending LLMs Against Cross-Model Adversarial Transfer via Representation Repulsion

Gal Wertheizer, Rom Himelstein, Tomer Peretz, Avi Mendelson

arXiv:2609.32602v1cs.LGcs.AI

TL;DR

Cross-model jailbreak transfer creates shared risk, and existing defenses are not designed for this threat. AnchorRep repels harmful-prompt representations from a frozen anchor using a lightweight LoRA adapter and no attack-specific training. Across five models, it reduces transfer ASR to 0–2% while preserving utility, though some attack classes partially bypass the defense.

  • Problem

    Adversarial prompts optimized on one open-weight LLM can transfer to architecturally different targets, while existing defenses are not designed for cross-model transfer.

    Source: §1 · p. 1

  • Method

    AnchorRep uses a lightweight LoRA adapter to repel the defended model’s harmful-prompt representations from those of a frozen anchor, without attack-specific training data.

    Source: §1 · p. 2

  • Results

    AnchorRep reduces transfer ASR to 0–2% across five models (7B–14B) while preserving utility, with reductions spanning attack types and cross-prompt GCG transfer.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support shaping shared representation geometry as a way to address cross-model transfer without attack-specific training.

    Source: §5 · pp. 8–9

    Embedding PGD retains 40–88% ASR on 14B models, while TAP reaches 42% versus a 50% baseline on Llama-3.

    Source: §4.2 · p. 7

Abstract

from arXiv · show

Adversarial attacks optimized on a single open-weight LLM can transfer to and jailbreak architecturally different models, allowing an attacker with white-box access to one model to compromise independently deployed systems. This creates a shared vulnerability across models, yet existing defenses are not designed for this cross-model threat. We find that cross-model transfer aligns with shared internal representation geometry, making it a natural defense target. AnchorRep targets this geometry directly with a lightweight LoRA adapter that pushes the defended model's internal representations of harmful prompts away from those of a frozen anchor model on the same prompts. Training uses a small set of harmful prompts and no adversarial examples. Across five models and four architectural families, AnchorRep reduces cross-model attack success rate to <=1.1% on 2,000 transferred attacks (0% on two), including the largest drop on Mistral (36% -> 1.1%). Existing defenses can reduce transfer, but only at high cost either inducing up to 77% degenerate benign output or increasing over-refusal by up to 18%. Because such degenerate benign outputs are not captured by standard refusal-based metrics, we introduce the Benign Garble Rate to quantify them. Our results suggest that cross-model robustness can be achieved by shaping representation geometry, without requiring attack-specific training

1 Introduction

Cross-model jailbreak transfer creates shared risk for independently deployed LLMs, while existing defenses can incur substantial benign-utility costs. AnchorRep targets the representational similarity associated with transfer, using a lightweight adapter and no attack-specific training.

1 Introduction

  • Cross-model attacks optimized on source models can compromise targets from different architectural families, creating shared risk as open-weight models proliferate.The attacks cited include GCG, PAIR, AutoDAN, and TAP.Source: §1 · p. 1
  • Existing defenses can achieve strong cross-model safety numbers but raise OR-Bench over-refusal by up to 18% or produce incoherent output on 77% of benign Llama-3 responses.The cited comparison is reported in Table 2.Source: §1 · p. 1
  • Higher within-family CKA correlates with higher transfer ASR, with the pattern replicated across 12 models in three architectural families using three similarity measures.Figure 1 shows pairwise similarity across 20 LLMs and the within-family CKA–GCG transfer ASR relationship.Source: §1 · p. 2, Figure 1 · p. 2, Table 1 · p. 2
  • AnchorRep repels harmful-prompt representations from those of a frozen anchor with a lightweight LoRA adapter, then uses auxiliary preservation losses to address benign-output degradation.Training uses no adversarial examples or attack-specific data; unconstrained CKA repulsion degrades benign output, quantified with the Benign Garble Rate.Source: §1 · p. 2
  • AnchorRep reduces transfer ASR to 0–2% across five models (7B–14B) while preserving utility, and extends across attack types and cross-prompt GCG transfer.The cross-prompt result is 51% →1.8% on Mistral.Source: §1 · p. 2

2 Related Work

Related work connects jailbreak transfer with representational similarity and adversarial-perturbation properties, while existing defenses primarily address same-model attacks. AnchorRep applies the similarity–transfer relationship as a defense mechanism.

2 Related Work

  • Existing defenses include input-level perturbation and representation engineering, but are not designed or evaluated for cross-model transfer.Embedding-space attacks have been shown to circumvent representation-engineering defenses [36], a limitation shared by AnchorRep.Source: §2 · p. 3
  • Diagnostic work links transfer to nearest-neighbor similarity [2], training-data overlap, and GCG token patterns [5] [20] [13], while AnchorRep turns the similarity–transfer relationship into an explicit defense mechanism.The cited findings are described as diagnostic rather than defense methods.Source: §2 · p. 3
  • Invariance regularization can improve robustness while degrading accuracy unless balanced by auxiliary objectives, a tradeoff AnchorRep addresses with losses that restrict reshaping to harmful prompts.The passage relates this tension to TRADES [44] and ARAT.Source: §2 · p. 3

3 Methodology

AnchorRep addresses a threat in which attackers optimize on a white-box source model without querying the distinct target, while defenders lack attack strings during training. It repels harmful-prompt representations from a frozen anchor and uses auxiliary objectives to preserve benign behavior.

3.1 Threat Model

  • The defender has no access to attack strings during training; in the cross-model setting, attackers optimize on a white-box source without querying the architecturally distinct target.The paper also evaluates an adaptive setting with direct white-box access to the defended target.Source: §3.1 · p. 3

3.2 Core Defense: CKA Repulsion

  • AnchorRep uses a lightweight LoRA adapter to minimize CKA between the defender and frozen-anchor representations on harmful prompts, pushing the defender away from shared transfer-mediating directions.CKA compares relational structure between models' representations and is dimension-agnostic; the prompt scope is a per-model hyperparameter.Source: §3.2 · pp. 3–4

3.3 Auxiliary Losses for Utility Preservation

  • CKA repulsion alone can reduce ASR to near-zero but degrades benign outputs at full-protection strength, so auxiliary objectives constrain where representations move.Figure 2 summarizes the repulsion and auxiliary-loss stages and reports harmful-prompt CKA dropping by 0.63 while benign CKA changes by 0.01 for the Q7/L3 pair.Source: §3.3 · p. 4, Figure 2 · p. 4
  • The refusal-direction loss steers harmful representations toward a precomputed refusal direction, while coherency anchors adapted representations near the base model.KL divergence enforces output-level invariance on benign and borderline prompts; an optional LM loss supplies reference refusal supervision for models with weak base refusal.Source: §3.3 · p. 4
  • The complete objective combines CKA repulsion, refusal-direction, coherency, KL, and LM losses; ablations show removing any component breaks the balance.Coherency and KL preserve benign behavior, while the refusal direction provides a stable target.Source: §3.3 · pp. 4–5

4 Experiments

Experiments evaluate AnchorRep across five models and four architectural families, testing cross-model transfer, adaptive attacks, utility, and the role of representation geometry. AnchorRep reaches the safety–utility target regime, while results also locate limits in attack coverage and identify a mid-depth intervention point.

4.1 Experimental Setting

  • Training uses 30 AdvBench harmful prompts, 500 WikiText-2 benign prompts, and 200 XSTest safe-subset borderline prompts for KL/coherency losses, with zero overlap against the held-out 2,000-prompt GCG transfer set.Source: §4.1 · p. 5

4.2 Results

  • Transfer ASR falls to 0–1.1% across five targets while BGR remains 0% and ∆OR stays ≤7.7%; only AnchorRep meets the stated safety–utility regime.Table 2 evaluates 2,000 transferred GCG prompts and defines the regime as ASR ≤2%, BGR = 0%, and ∆OR ≤8%.Source: Table 2 · p. 6, §4.2 · p. 6
  • On HarmBench, aggregate ASR drops from 5.3% to 2.7%; on FalseReject, three of five defenders shift by ≤5% from baseline, while Vicuna shifts +12.8%.Mistral’s HarmBench ASR drops from 11.8% to 2.0%.Source: §4.2 · p. 6
  • Despite no adversarial training, AnchorRep reduces ASR in 22 of 25 white-box attack–target settings, with strongest reductions on discrete-token attacks and smaller-model Embedding PGD.Table 3 reports defended-minus-baseline ∆ASR for five attacks optimized directly on each target.Source: §4.2 · pp. 6–7, Table 3 · p. 7
  • Embedding PGD retains 40–88% ASR on 14B models, while TAP partially bypasses the defense on Llama-3 at 42% versus a 50% baseline.These results delimit same-model robustness for continuous-space and semantic-reframing attacks.Source: §4.2 · p. 7
  • Across the CKA-weight sweep, all settings eliminate transfer at approximately 0% ASR, but only intermediate γ preserves utility; strong γ collapses benign representations to CKA 0.142.Removing CKA yields ∆OR = +32.4%, weak γ yields ∆OR = +76.4%, and strong γ yields ∆MT = −2.34; wider loss perturbations break at least one utility constraint.Source: §4.2 · p. 7, Table 4 · p. 7
  • The 50%-depth intervention is the only tested layer point that achieves near-zero ASR with zero BGR across all five models; shallow layers garble outputs and deep layers leave attacks effective.The reported failure cases include 12.2% BGR at 25% depth and up to 20% ASR at 62.5% depth.Source: §4.2 · pp. 7–8
  • On Mistral-7B, verbatim GCG suffixes transfer across unrelated prompts at 51.3% baseline, but AnchorRep reduces this rate to 1.8% across the same conditions.The authors interpret the low variation across donor and target categories as evidence of a direction common to successful suffixes.Source: §4.2 · p. 8

5 Discussion

The discussion frames cross-model transfer and defense through shared representation geometry and distinguishes hollow compliance from benign garbling. It also details representation-similarity measurements, transfer patterns and asymmetries, the evaluation pipeline and Stage 5 judge choice, and appendix analyses of interventions, transfer, baselines, and representation extraction.

5 Discussion

  • Cross-model transfer tracks shared representation geometry, which AnchorRep repels using harmful prompts alone; weakly optimized and high-loss suffixes transfer comparably.The discussion reports that residual ASR is uniform across donor prompts, categories, and optimization levels, consistent with disrupting a shared direction rather than suffix-specific features.Source: §5 · pp. 8–9
  • AnchorRep can produce “hollow compliance”: fluent, on-topic benign responses that lack actionable content, distinct from refusal and incoherent garbling.The proposed mechanism preserves language modeling, style, and instruction following while disrupting representations that ground harmful requests in actionable responses.Source: §5 · p. 9
  • Only the 50th-percentile layer achieves near-zero ASR with near-zero BGR; shallower layers degrade utility, deeper layers fail to suppress transfer, and multi-layer strategies are suboptimal.The authors interpret this depth window as evidence that the compliance-inducing direction is localized to a residual-stream band, consistent with prior work [3] [39].Source: §5 · p. 9

A.1 Jailbreak Classification Pipeline

  • The jailbreak classifier combines heuristic filters with WildGuard [14] for borderline cases, achieving F1 0.778 and accuracy 0.836 in the judge benchmark.The pipeline filters degenerate and gibberish outputs before scoring compliance, coherence, and refusal, then applies automatic decisions or a WildGuard tiebreaker.Source: §A.1 · pp. 14, 16

A.2 Benign Garble Rate (BGR)

  • All five AnchorRep defenders achieve 0% BGR on OR-Bench, while BGR measures garbled benign responses that refusal-based metrics can miss.The paper reports that competing-defense flags were manually verified as genuine repetition-loop degeneracies.Source: §5 · p. 9, §A.2 · p. 16, Table 6 · p. 18

A.3 GCG Attack Configuration

  • Transfer GCG uses 500 optimization steps per prompt, 100 prompts per source, and 20-token suffixes; existing defenses are retrained on Llama-3-8B and Mistral-7B for uniform comparison.CRL on Mistral uses the original paper’s hyperparameters because the codebase-default configuration caused 100% BGR.Source: §A.3 · p. 16, §A.4 · p. 17

A.5 Absolute Benchmark Values

  • Appendix benchmark tables report undefended baselines, defended absolute scores, and per-source transfer ASR for defended Mistral-7B and Llama-3-8B across 2,000 GCG prompts.Table 7 groups sources as self, anchor, and other; its counts aggregate to the same Self/Transfer values as the main results.Source: Table 5 · p. 17, §A.6 · p. 17, Table 6 · p. 18

A.7 CKA: Construction and Implementation Details

  • CKA uses the final prompt-token hidden state at the target layer across all five defenders, supplying representations for repulsion, coherency MSE, and refusal-direction objectives.The appendix says this extraction outperformed mean-pooling and first-token reductions in preliminary checks.Source: §A.7 · pp. 18–19
  • Representation-space losses use the same target layer at 50th-percentile depth, whereas token-level cross-entropy and KL losses operate on logits.The shared layer setting applies to CKA repulsion, coherency, and refusal-direction losses.Source: §A.7 · p. 19

A.8 Alternative Similarity Metrics

  • Distance Ratio measures whether models separate harmful and benign prompt clusters similarly, regardless of whether those regions have similar orientations.The analysis also compares representation-similarity metrics with transfer ASR across model pairs, separately for same-family and cross-family pairs.Source: §A.8 · pp. 19–20

A.9 Full GCG Transfer ASR Matrix

  • Three representationally coherent families—Mistral, Llama, and Eastern—were used for family-level analysis, while Gemma and DeepSeek were excluded for weak similarity to their nominal groupings.Phi-2 and Baichuan2 were included with Llama based on representation similarity rather than lineage.Source: §A.9 · p. 21
  • ρ = +0.783 (p < 0.001) with Hermes-2 included, versus +0.910 (p < 0.001) when excluded, as directional transfer asymmetry adds noise to the correlation.Hermes-2 suffixes transfer to Zephyr and NeuralChat at 58% and 75%, while their suffixes transfer to Hermes-2 at 11% and 13%, respectively.Source: §A.9 · p. 22
  • ρ = +0.910 (p < 0.001): same-family pairwise CKA (mid-layer, harmful prompts) correlates with transfer ASR, and same-family transfer is higher on average.The matrix covers 19 aligned models; Llama-3 and Phi-2 have ≤1% ASR as targets from every source, while NeuralChat has 60–82%.Source: §A.9 · p. 21, Table 10 · p. 22

A.10 WildGuard Prompt Format

  • F1 0.778 vs. 0.720 and accuracy 0.836 vs. 0.812: Pipeline + WildGuard [14] outperformed Llama-Guard-3-8B as the Stage 5 tiebreaker.The benchmark motivated switching to WildGuard with the original harmful prompt supplied alongside the response.Source: §A.10 · p. 23

A.11 Jailbreak Judge Benchmark

The benchmark selects a heuristic pipeline with WildGuard as its final judge, since it performs best overall among pipeline variants. Standalone judges have important error modes, and symmetric manual verification preserves the reported defense gap.

A.11 Jailbreak Judge Benchmark

  • HarmBench-13b has the highest standalone recall (0.800) but false-positive rates of 28% on refusals and 13% on garbled outputs; WildGuard misclassifies 71% of garbled outputs as unsafe.LlamaGuard has the best standalone precision (0.847) but lowest recall (0.555).Source: §A.11 · pp. 23–24, Table 11 · p. 23, Table 13 · p. 24
  • Every pipeline configuration has 0% false positives on garbled samples, which the heuristic stages filter before neural judging.The pipeline uses heuristics followed by a neural judge as a tiebreaker.Source: §A.11 · pp. 23–24
  • F1 0.778, accuracy 0.836, and precision 0.847 make Pipeline + WildGuard the strongest overall pipeline variant, with recall 0.720.The authors use this configuration for all main-paper experiments.Source: §A.11 · p. 24, Table 12 · p. 24
  • The symmetric manual audit preserves a positive defense gap for every defender, with Mistral’s gap remaining approximately 35 percentage points.The raw automated baseline closely approximates verified vulnerability after correcting both flagged responses and judge-missed jailbreaks.Source: §A.11 · pp. 24–25, Table 15 · p. 25

B Manual Verification Protocol

The protocol manually reviews every defended-model response flagged as a jailbreak and overturns a flag only for non-actionable hallucination, satire or physical impossibility, or an immediate refusal pivot. Audits distinguish these artifacts from genuinely harmful responses and reveal different utility failures across defenses.

B Manual Verification Protocol

  • A flagged response is overturned only for non-actionable hallucination, satire or physical impossibility, or an immediate pivot to refusal or prevention advice.Actionable harmful content remains a positive, even when partially garbled or caveated; ambiguous cases are also left positive.Source: B Manual Verification Protocol · pp. 25–26
  • Existing defenses’ flagged responses contained genuinely harmful content and were not overturned; AnchorRep’s flagged responses were almost uniquely marked by non-actionable hallucinations.Other defenses instead showed over-refusal or output degradation, captured separately by OR-Bench and BGR.Source: B Manual Verification Protocol · p. 26
  • 52 of Circuit Breakers’ 58 flagged Llama-3 responses were gibberish and overturned, yielding approximately 0.5% manually verified ASR (10/2,000) versus 2.9% raw automated ASR.The symmetric audit also recovered coherent compliances missed by the automated judge.Source: B Manual Verification Protocol · p. 26

C Evolution of the Defense Mechanism

The defense evolved through extensive configuration testing toward differentiable CKA repulsion and a single-anchor design. The selected approach accounts for model-specific sensitivity, while tested alternatives exposed instability or utility costs.

C Evolution of the Defense Mechanism

  • CKA’s differentiable matrix formulation supports end-to-end backpropagation in the joint LoRA update without surrogate gradients or alternating optimization.This excludes non-differentiable matching approaches that would require such alternatives.Source: C Evolution of the Defense Mechanism · p. 26
  • Mistral required a 4× lower learning rate, higher KL regularization (ε = 0.8 vs. 0.4), and lower CKA repulsion (γ = 0.7 vs. 2.0) after identical training produced 30–100% garble rates.Reducing its learning rate from 2 × 10−4 to 5 × 10−5 restored normal weight magnitudes.Source: C Evolution of the Defense Mechanism · p. 26
  • Across 20 retraining runs using four independently sampled 30-prompt harmful subsets per model, max(s, a, o) ASR had per-model standard deviation at most 1.91%, with worst-case verified ASR of 5%.All main-paper results use seed 42.Source: C Evolution of the Defense Mechanism · p. 26
  • Pairwise, squared, rebalanced, and multi-layer CKA alternatives did not improve on single-layer batch CKA; benign CKA preservation reduced ASR but caused 16.9% OR-Bench garble.The authors attribute the latter to unstable gradients from conflicting harmful-minimization and benign-maximization objectives.Source: C Evolution of the Defense Mechanism · p. 27
  • Same-family, cross-family, and mixed multi-anchor configurations all became unstable or collapsed benign outputs, so the released method uses a single anchor.No multi-anchor configuration completed training with both stable convergence and acceptable utility.Source: C Evolution of the Defense Mechanism · p. 27

D Anchor Model Ablation

Anchor selection, intervention depth, and the number of perturbed layers all affect the safety–utility tradeoff. Across the ablations, the best choice varies by model, while single-layer intervention avoids failures seen in some alternatives.

D Anchor Model Ablation

  • Across anchors, lower defender–anchor RSA similarity correlates with larger relative ASR reduction (ρ = +0.39), but within-family the relationship reverses (ρ = −0.50); no anchor is universally optimal.BGR remains ≤1.1% across anchor choices; the ablation table uses preliminary hyperparameters, so its selected rows underestimate final tuned performance.Source: D Anchor Model Ablation · pp. 27–28, Table 16 · p. 27

E.1 Layer-Depth Mechanism: The Safety–Semantic Boundary

  • Intervening at Mistral’s 43–47% depth disrupts safety representations and makes the model unusable, while later intervention misses the safety subspace and can increase attack compliance.The section places the 50% layer at the transition between safety encoding and semantic output.Source: §E.1 · p. 28

E.2 Cross-Model Generalization of the Layer Pattern

  • Layer sensitivity varies by model: Llama-3 shifts from 21/22/13% ASR at layer 15 to 1/0/2% at layer 16, while Qwen-14B stays at 0–3% across layers 37.5–55%.The Mistral Table 17 sweep used an earlier exploratory configuration; its production configuration showed 0% BGR at the same layers.Source: §E.2 · pp. 28–29, Table 17 · p. 29
  • Phi-3’s OR-Bench BGR rises from 0% to 34.8% at layer 45% and 48.1% at 47.5%, despite 0–1.1% BGR on the smaller evaluation set.This highlights why layer choices should be checked across diverse benchmarks.Source: §E.2 · pp. 28–29

E.3 Multi-Layer Strategies

  • The single-layer approach reduces ASR by 22% and improves MT-Bench by 0.03, while multi-layer approaches either fail to reduce ASR or incur substantial MT-Bench degradation.The best multi-layer result has +8% ASR change and 5.6% BGR; the authors hypothesize that spreading repulsion dilutes each layer’s gradient.Source: §E.3 · p. 29

E.4 LoRA Layer Subset Ablation

Restricting LoRA to a subset of layers weakens the defense substantially, even when the subset includes the CKA target layer. Full-network LoRA is needed to disrupt transfer, at a cost to utility.

E.4 LoRA Layer Subset Ablation

  • 12–44% ASR across restricted configurations compares with 1/0/2% using all layers, and the table identifies the 50% layer as the production setting.Even the “mid 9” configuration reaches 31/38/44% ASR.Source: §E.4 · p. 30, Table 20 · p. 31
  • No multi-layer CKA strategy outperforms single-layer CKA at the 50th percentile in the reported Mistral ablation.The table uses an earlier Mistral configuration, so its values differ from the canonical production configuration.Source: Table 19 · p. 31
  • 31/26/31% ASR and 16.1% OR-BGR with LoRA only on layers 0–15 show that adapting layers below the CKA target is insufficient.Early-layer inputs remain in their pretrained geometry, while output layers must also adapt to the reshaped mid-layer representations.Source: §E.4 · pp. 30–31
  • Restricted LoRA improves MT-Bench to 6.44–6.58 from 6.42 and lowers over-refusal to 2.9–30.2% from 38.5%, trading utility gains for weaker security.The authors attribute transfer disruption to perturbation across all layers.Source: §E.4 · p. 31

F.2 Per-Category Generalization

AnchorRep generalizes across harm categories and cross-prompt GCG suffixes, while the appendix also examines adaptive attacks and qualitative failure modes. The Circuit Breakers comparison highlights how turn-boundary errors can distort judged generation scores.

F.2 Per-Category Generalization

  • −3.6, −6.5, and −8.0 are the reported reductions for chemical/biological, cybercrime, and harassment; copyright is the exception at +0.5.The authors attribute the copyright result to memorized content being stored in model weights rather than the targeted harmful representation subspace.Source: §F.2 · pp. 31–32, Table 22 · p. 32

F.3 Cross-Prompt Suffix Universality

  • Cybercrime suffixes transfer to illegal targets at 79% and misinformation at 100%, versus 48% within cybercrime; suffixes with GCG loss ≥1.5 reach 71.3% cross-prompt ASR versus 46.3% for low-loss suffixes.The authors hypothesize that less-optimized suffixes find broader adversarial directions, while over-optimized suffixes overfit to source-prompt tokens.Source: §F.3 · p. 32, Table 23 · p. 33
  • 51.3% to 1.8% cross-prompt transfer: AnchorRep reduces transfer across donor categories and loss levels, and blocks early-stopping suffixes at every tested step count (0.0% DEF ASR).On baseline Mistral, a 10-step suffix transfers at 30%; additional optimization raises transfer only to 36% from 10 to 500 steps.Source: §F.3 · p. 33, Table 24 · p. 33
  • Baseline cross-prompt ASR is <1% on Llama-3-8B and Vicuna-7B, 5.3% on Qwen-1.5-14B, and 58% on Mistral-7B.The authors suggest cross-prompt suffix universality reflects insufficient safety training rather than a fundamental property of transformer representations.Source: §F.3 · p. 33

F.4 Adaptive Attacks

  • The adaptive-attack appendix reports absolute ASR for five white-box attacks, with 100 prompts per attack, on both undefended and defended models.The main text reports deltas for these attacks.Source: §F.4 · p. 33, Table 25 · p. 34, Table 26 · p. 34

G.1 Defended Model Responses

  • AnchorRep’s characteristic failure mode, “hollow compliance,” is coherent but non-actionable output that can superficially resemble compliance to automated judges.Examples include treating GCG suffix tokens as nouns, inventing impossible devices, or pivoting to safety advice.Source: §G.1 · p. 34, Table 27 · p. 35

G.2 Existing Defense Failure Modes

  • The appendix pairs examples of existing-defense failures on benign prompts with representative classifications from successive stages of the jailbreak-detection pipeline.Earlier pipeline stages take precedence over later ones.Source: §G.2 · p. 35, §G.3 · p. 36, Table 30 · p. 37

H.2 Three-way Comparison: Baseline, Public CB, Retrained CB

  • The public Circuit Breakers checkpoint scores 7.20 on MT-Bench versus 6.44 for baseline, while the retrained checkpoint has a −0.18 ∆MT; the public checkpoint’s turn-marker overshoot can inflate judged scores.For the coding example, both models produce identical initial code, but the local judge scores the longer CB response 9 versus 5.Source: §H.2 · p. 39, §H.3 · p. 39, §H.4 · p. 39

H.5 Discussion

The public Circuit Breakers checkpoint’s elevated MT-Bench score is attributed to a tokenizer misconfiguration that causes turn-boundary overshoot, not improved generation quality. The appendix then documents training data and hyperparameter choices for AnchorRep.

H.5 Discussion

  • The public Circuit Breakers checkpoint’s elevated MT-Bench score is attributed to tokenizer misconfiguration and turn-boundary overshoot, while longer responses can receive higher scores from local judges regardless of substantive quality.The authors note that local non-GPT-4 judges are especially susceptible to length bias.Source: §H.5 · pp. 39–40
  • The public checkpoint remaps eos_token to <|end_of_text|>, whereas the retrained checkpoint preserves <|eot_id|> at turn boundaries during training.The authors propose that the public training pipeline may therefore have omitted gradient signal to emit the turn-level terminator.Source: §H.5 · p. 40

I Training Prompts

  • Training uses 30 harmful prompts sampled from AdvBench and 10 refusal templates; the 30 prompts are disjoint from the GCG transfer and HarmBench evaluation prompts.The authors state that this split supports genuine generalization rather than memorization of training prompts.Source: I Training Prompts · pp. 40–41

J.1 Universal Defaults

  • Across successful configurations, AnchorRep uses CKA alignment, LoRA rank 32, a 50th-percentile target layer, and includes borderline prompts for KL preservation rather than CKA repulsion.The refusal-direction weight is 0.1 for Llama-3 and 0.15 for the other defenders; the coherency weight is 1.0.Source: §J.1 · p. 42

J.2 Model-Specific Gamma (γ)

  • The anchor-repulsion weight γ is model-sensitive: the authors recommend starting at γ = 1.0 and adjusting based on initial defender–anchor CKA, using BGR as an early-stopping signal.Greater cross-family representational distance may require higher γ, while closely related architectures may garble at higher values.Source: §J.2 · p. 42

J.3 Regularization: ε (KL) and δ (LM Loss)

  • KL loss preserves the output distribution, while LM loss preserves next-token quality; with both set to zero in a Mistral ablation, MMLU falls from 62.5% to 41.2%.That ablation still achieved XSTest 6.0% and OR-Bench 6.7%.Source: §J.3 · p. 42

J.4 CKA Scope and Training Steps

The appendix examines how training scope, loss weights, anchor choice, learning rate, and borderline-prompt source affect the defense. It also reports CKA cost and sensitivity to estimator and batch size, with empirical validation at 70B scale left for future work.

J.4 CKA Scope and Training Steps

  • Mistral-7B requires about 50% more training steps than other 7B models across CKA scopes, which the authors attribute to its greater sensitivity to representation perturbation.For scope=all, roughly 60% more samples contribute CKA loss per batch, so the authors double training steps for comparable convergence.Source: §J.4 · p. 43

J.5 One-at-a-time Hyperparameter Perturbation

  • Every one-at-a-time perturbation of the picked Mistral loss weights breaks at least one utility budget; only the picked configuration satisfies all four simultaneously.The tested budgets are |∆OR| > 8%, |∆XS| > 8%, |∆MT| > 0.5, and OR-BGR > 4%.Source: §J.5 · p. 43, Table 34 · p. 44

J.6 Anchor Selection

  • Cross-family anchors consistently outperform within-family anchors in the reported comparisons; for Mistral, Llama-3 anchors reduce OR-Bench refusal by 40–50% relative to Llama-2 while maintaining comparable ASR.The authors recommend an architecturally distinct anchor to maximize representational contrast.Source: §J.6 · p. 43

J.7 Learning Rate Sensitivity

  • Reducing Mistral’s learning rate fourfold restores LoRA-B weight magnitude and eliminates garbling while preserving defense effectiveness (ASR ≤4%, BGR = 0%, MT-Bench ≥6.2).At the standard rate, post-training LoRA-B weights are about 4× larger than in other models under equivalent configurations.Source: §J.7 · p. 44

J.8 Borderline Prompt Source

  • Switching borderline prompts from XSTest to WildGuard increases LoRA-B magnitude about 4× and raises benign garble rate from 0% to 29% in the controlled Mistral comparison.The authors use XSTest borderline prompts in final configurations and evaluate on OR-Bench and WildGuard to avoid train–test contamination.Source: §J.8 · p. 45

K.1 CKA Training Cost

  • In our experiments, CKA computation added <5% wall-clock overhead to each training step, with total training taking 15–30 minutes for 200–600 steps on a single NVIDIA L40S GPU (48 GB).For a 70B model, the authors estimate that mini-batch CKA (N ′ = 8) and layer-sparse extraction could fit within 4× A100-80GB GPUs and the same 200–400 step budget, adding <10% overhead to standard LoRA fine-tuning; empirical validation at this scale is left to future work.Source: §K.1 · p. 45, §K.2 · p. 46

K.3 Biased vs. Debiased CKA Estimator

  • The debiased estimator preserves similar ASR but shifts utility: OR-Bench refusal drops from 67.9% to 1.6%, BGR rises from 0% to 6.1%, and MT-Bench drops from 6.62 to 6.21.The authors recommend the biased estimator at operating batch size N=4 for its utility profile.Source: §K.3 · p. 46, Table 35 · p. 46

K.4 Batch Size N Sensitivity

  • ASR stays at or below 2% for batch sizes N=4, 8, and 16, while larger batches degrade MT-Bench and inflate benign XSTest refusals.The authors select N=4 because it concentrates the CKA gradient on harmful prompts and better preserves benign representations.Source: §K.4 · p. 46, Table 36 · p. 47

K.5 Training-Subset Variance

The appendix probes training-subset stability, evaluates over-refusal on a separate safe-but-sensitive prompt set, and tests transfer from multiple attack sources to defended Llama-3.

K.5 Training-Subset Variance

  • Across 20 runs, worst-case verified ASR was 5%, and per-model standard deviation of max(s, a, o) across seeds was at most 1.91%.The runs used four harmful-prompt training sets; every flagged jailbreak was manually verified.Source: §K.5 · p. 47, Table 37 · p. 47
  • The seed-42 main-paper split has zero train/test overlap, while the other seeds re-sample training prompts without re-partitioning GCG evaluation prompts.Those other seeds are reported only as a robustness probe; every headline result uses seed 42.Source: §K.5 · p. 47

K.6 Clean Over-Refusal: FalseReject Evaluation

  • The FalseReject evaluation uses 500 safe-but-sensitive prompts with no overlap with XSTest, OR-Bench, AdvBench, or WikiText-2.The supplied passages describe the evaluation set but do not report its refusal-rate results.Source: §K.6 · p. 47, Table 38 · p. 48

K.7 Ensemble-Source GCG Transfer

  • 0% ASR: all three ensemble-source GCG variants achieved 0% ASR on defended Llama-3.The suffix with lower loss across Mistral-7B and Qwen-1.5-7B was selected for each prompt in the Ensemble row.Source: §K.7 · pp. 47–48, Table 39 · p. 48

L Limitations

AnchorRep’s same-model adaptive robustness is limited for some attack types, and its single-anchor design, parameter choices, and scaling impose further boundaries.

L Limitations

  • Continuous embedding-space attacks bypass AnchorRep, while TAP’s semantic-reframing attack partially circumvents it on Llama-3.The defense reduces ASR for several discrete-token attacks, including GCG, PAIR, and AutoDAN.Source: L Limitations · p. 48
  • A defense-aware attacker jointly optimizing across surrogate geometries is not examined and may partially circumvent single-anchor repulsion.The paper identifies an attacker who knows both the defender and anchor and avoids the anchor’s trajectory as a possible scenario.Source: L Limitations · p. 48
  • Optimal γ varies across defender–anchor pairs, and sensitivity to substantially different refusal-template styles or languages has not been examined.Anchor scale and training-data overlap may also affect defense strength beyond the ablation coverage.Source: L Limitations · p. 48
  • Scaling to 70B+ is described as feasible with fewer than 5% overhead from CKA Gram matrices, but requires multi-GPU sharding for the forward pass.Source: L Limitations · p. 48

M Deployment Overhead and Reproducibility

This section describes AnchorRep’s deployment overhead and reproducibility resources, including released defensive adapters, artifacts, and supporting documentation. It also reports the defense’s scope limits, variability and dual-use concerns, asset licensing, the absence of human-subjects research, and the LLM components used in the methodology.

M Deployment Overhead and Reproducibility

  • After merging the LoRA adapter, the defended model matches the base model’s architecture, parameter count, inference latency, and memory footprint, with no extra runtime modules or checks.Training takes 15–30 minutes on one NVIDIA L40S GPU (48 GB), using 200–600 gradient steps; the anchor is freed before training.Source: M Deployment Overhead and… · p. 48, §2 · p. 54
  • Greedy decoding, fixed seed 42, and evaluation on 2,000 transferred prompts support reproducibility, while seven independently trained defenders provide cross-model replication.The paper reports hyperparameters and per-source ASR breakdowns, releases manual-verification audit logs, and provides code, configurations, evaluation artifacts, and adapters; headline results use single runs, with training-subset variance reported separately.Source: M Deployment Overhead and… · pp. 48–49, §2 · pp. 52–53

N Asset Licenses and Access

  • The paper catalogs exact assets, license terms, and access information for datasets, defenders, anchors, the judge classifier, and 20 source models, noting inherited or combined terms where applicable.Four source models also appear in the defender/anchor list and are not duplicated.Source: N Asset Licenses and Access · p. 49, Table 40 · p. 49

2. Limitations

  • The defense targets cross-model transfer, not same-model adaptive embedding-space attacks; single-anchor repulsion can be challenged using multiple surrogates, γ depends on the defender–anchor pair, and non-English prompts and refusal templates are untested.Embedding PGD and TAP are identified as attack classes that partially bypass the defense.Source: §2 · p. 51
  • Across four independent samples of the 30-prompt training set, per-model standard deviation of max(s, a, o) ASR is at most 1.91% across five defenders.Representation-similarity correlations are accompanied by p-values from a two-sided permutation test on the 20-model pair set; uncertainty is reported as 1σ.Source: §2 · p. 54
  • The paper describes the work as defensive but notes that released GCG evaluation artifacts are dual-use and that shared compliance-related representations could enable defense-aware attacks.It proposes releasing evaluation protocols and audit logs, considering multi-anchor variants, and monitoring degenerate outputs with BGR.Source: §2 · p. 55
  • Released adapters are described as reducing harmful compliance, while evaluation artifacts use standard public attack methods and are accompanied by defensive-use guidance and evaluation-only scripts.The artifacts include GCG suffixes for AdvBench and HarmBench evaluations and HarmBench configurations.Source: §2 · p. 56
  • The paper says its datasets, base models, anchors, and reference defenses are credited, publicly available under their respective licenses, and used according to their terms.License names, detailed references, and access links are provided in Appendix Section N and Table 40.Source: §2 · pp. 56–57
  • The release includes AnchorRep LoRA adapters for five models, training/evaluation code, and a manual-verification audit log, with documentation of hyperparameters and limitations.The combined adapter size is approximately 1.85 GB, so the adapters are hosted in a companion HuggingFace Collection and downloaded by evaluation scripts on demand.Source: §2 · p. 57
  • The paper reports no crowdsourcing or human-subjects research; manual verification was performed by the authors on model-generated text, so IRB review does not apply.Source: §2 · pp. 57–58
  • LLMs are integral to the methodology: defended models, frozen anchors, and the WildGuard judge are used in training and evaluation, with components identified and configured in the paper.WildGuard supports automated safety assessment of model outputs.Source: §2 · p. 58

Related papers

Loading 2609.32602v1…