PaperlayerView exact revision on arXiv

Source-linked AI summary

Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking

Marco Biroli

arXiv:2609.32116v1cs.AIcond-mat.stat-mech

TL;DR

Best-of-N attack curves leave unexplained why their fitted exponents take particular values, and the full two-budget surface has received little study. This paper models prompts and augmentations as thermal barriers; it finds a drifting exponent and an apparent exponential crossover tied to finite dataset size.

  • Problem

    Curve fitting does not explain why BoN attack exponents take their values, and the full (N, M) surface and temperature dependence have received little study.

    Source: §1 · p. 1

  • Method

    A physical barrier model assigns prompts entropic bottlenecks and augmentations random thermally activated saddles to describe the attack surface.

    Source: §1 · p. 2

  • Results

    The local exponent rises from 0.1 at N = 2 to above 2 at N ∼10^2–10^3 without a plateau, and the reported exponential crossover is a finite-dataset artifact.

    Source: §4.3 · p. 8, §6 · p. 9

  • Takeaways & Limitations

    The model quantitatively compares model safety and makes predictions for defensive practices.

    Source: §6 · p. 9

    The study covers only intermediate-size open-weight models and one dataset of 400 adversarial prompts, limiting large-N accuracy and claim universality.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Best-of-$N$ jailbreaking (BoN) bypasses safeguards of aligned models by drawing $N$ independent augmentations of an unsafe prompt and sampling $M$ completions of each. Previous works have shown that the attack success rate (ASR) seems to follow a power-law in $N$, which we challenge. The exponent drifts with $N$, with an exponential crossover which is a finite-size artifact of the adversarial dataset. Little work has been done to explore the entire two-budget ($N, M$) attack surface as well as its dependence on the generation temperature $T$. We introduce a simple barrier model where each prompt has a baseline safety level and each augmentation a random thermally activated barrier. Then four numbers, each backed by an interpretable safety mechanism, determine the entire ($N, M$) attack surface. They extrapolate predictions from $N \leq 100$ to $N = 10^4$, collapse five distinct models on the same scaling function and predict ASR at different temperatures from the one they were fitted at.

1 INTRODUCTION

Best-of-N attack curves are used to estimate model resistance, but curve fitting does not explain the exponent or where failures originate. The paper proposes a physical model of the two-budget attack surface and predicts that ASR does not follow a clean power law.

1 INTRODUCTION

  • Best-of-N attack curves estimate resistance, but fitting them does not explain why the exponent takes its value or where failures stem from.The paper identifies curve fitting as conflating distinct sources of randomness.Source: §1 · p. 1
  • The model treats safety as an entropic bottleneck and generation at temperature T as thermal activation across an energetic barrier.The proposed safety boundary is a single hyperplane in an approximately linear residual space.Source: §1 · p. 2
  • The model predicts that ASR does not follow a clean power law, contrary to established reports.It is intended to reproduce the full two-budget attack surface rather than only fit benchmark curves.Source: §1 · pp. 1–2

2 BACKGROUND AND RELATED WORK

Prior BoN analyses primarily fit benchmark curves, while the full (N, M) surface and its temperature dependence have received little study. The paper distinguishes how prompt-level and augmentation-level variability shape those attack budgets.

2 BACKGROUND AND RELATED WORK

  • Prior work reports power-law ASR growth and an eventual exponential crossover, but has not studied the full (N, M) surface or temperature dependence.Earlier ablations found multiple generations did not improve attack efficiency.Source: §2 · p. 2

Power laws

  • Feng et al. (2026) derive an exact N power law from a Beta-distributed per-prompt rate, whereas this model's across-prompt law is log-normal.The log-normal left tail yields a slowly varying exponent rather than a fixed one.Source: §2 · pp. 2–3
  • In this model, augmentation-level variation shapes the M channel, while across-prompt variation sets the N channel.At M = 1, the augmentation-level distribution enters only through its mean.Source: §2 · p. 3

Glassy systems

  • Unlike Halder et al. (2026), this model resolves both attack budgets and assigns different exponents to the two channels.Halder et al. model generated tokens as degrees of freedom in token-space energy landscapes.Source: §2 · p. 3

3 THE MICROSCOPIC MODEL

The microscopic model represents unsafe generation as thermal escape across an entropic bottleneck and augmentation-dependent energetic saddles. Four parameters describe its attack surface, with a large-budget asymptote that has a running rather than fixed power-law exponent.

3 THE MICROSCOPIC MODEL

  • Four parameters, µα, σα, µβ, and σβ, describe the entire (N, M) attack surface.The model's baseline barriers are Gaussian-distributed, while prompt-specific saddle-height statistics are log-normal-distributed.Source: §3 · p. 3
  • The model assumes a Gaussian prompt-level entropic barrier and prompt-specific, exponentially distributed saddle heights selected by augmentations.Generation crosses a saddle with Arrhenius probability at temperature T.Source: §3 · p. 3
  • For large barriers and M e^-αb ≪ 1, averaging over augmentations gives an effective barrier with Rb(M) ≃ M e^-Eb.The approximation treats M completions as parallel jumps over the effective barrier.Source: §3 · p. 4
  • The large-N asymptote is not a clean power law: its effective exponent varies slowly with N.The supplied asymptote scales as 1/(γ^2 log z).Source: §3 · p. 4
  • The large-N saddle-point approximation does not provide the low-budget asymptote; that regime requires expanding the underlying expression directly.The low-budget case is treated separately for small N.Source: §3 · p. 4

4 EXPERIMENTAL VALIDATION

Across three families of open-source models, experiments support the barrier model and its four-parameter description of Best-of-N attack behavior. The model reconstructs and extrapolates attack success, captures temperature and scaling effects, and explains the apparent large-budget exponential crossover as a finite-dataset effect.

4 EXPERIMENTAL VALIDATION

  • Across prompts, αb is Gaussian and βb is log-normal; after centering and standardizing, the measured distributions cannot be distinguished across the instruct models.The normal and log-normal families are universal, while their moments are model-specific.Source: §4 · p. 5, Figure 2 · p. 6
  • Defense mechanisms differ across models: Gemma favors an entropic α channel, whereas Llama is aligned via an energetic β channel.Instruct tuning raises the entropic barrier relative to the base model.Source: §4 · p. 6
  • A safety direction separates safe and unsafe completions with held-out AUC 0.89, and an augmented-prompt probe recovers the saddle with held-out R2 = 0.69.The saddle projection’s spread is inversely correlated with βb (Spearman −0.59).Source: §4 · p. 6, Figure 3 · p. 6

4.1 RECONSTRUCTION AND EXTRAPOLATION

  • The four-parameter model fitted on N ≤100, M ≤30 closely predicts larger attacks; it matches Feng et al. (2026) on Qwen3B and is more accurate on Qwen7B.The power-law extrapolation to N ∼10^4 is significantly off.Source: Figure 4 · p. 7, §4.1 · p. 7
  • MAE is 0.031 for the four-parameter population model and 0.034 for the per-prompt fit, despite using 4 versus 800 fitting parameters.The comparison evaluates reconstruction of measured attack success on the N, M ≤100 grid.Source: §4.1 · p. 7
  • Eq. (21), fitted at T = 1, reduces prediction error by a factor of 3.7 to 7.5 versus pure Arrhenius at all four unseen temperatures.The entropic barrier stays constant with temperature, while the saddles scale as 1/T.Source: Figure 5 · p. 7, §4.1 · p. 8

4.2 SCALING COLLAPSE

  • Across models, the AFR collapses onto a shared scaling-function family in z = (NM)e^-μE, with a distinct separation between groups of models.The closed-form scaling is valid only for M ≪ e^μα; the M-agnostic form remains accurate at large z for all M.Source: §4.2 · p. 8, Figure 6 · p. 8

4.3 POWER-LAW AND FAKE EXPONENTIAL CROSSOVER

  • The local exponent drifts from 0.1 at N = 2 to above 2 at N ∼10^2–10^3, then the finite dataset’s hardest prompt drives an apparent exponential crossover.For 400-prompt AdvBench, Nres is approximately 10^2 for Qwen-3B-Instruct and ≈2 × 10^3 for Qwen-7B-Instruct; adding harder prompts shifts the bend right.Source: §4.3 · p. 8, Figure 7 · p. 9

5 SAFETY CONSEQUENCES

The model links defense choices to the distribution of safety barriers and to the relative cost of augmentations and completions. Greater barrier variability trades weaker protection at smaller budgets for stronger protection at large budgets.

5 SAFETY CONSEQUENCES

  • When completions are cheaper, the attacker should use M ⋆∼e^µα; when augmentations are cheaper, M ⋆= 1 is optimal.The model places diminishing returns on additional attempts once M approaches e^µα.Source: §5 · p. 9
  • At large attack budgets, higher σE is associated with safer models, while at smaller budgets it is associated with weaker models.BoN eventually reaches any barrier height; high variability makes the few barriers far above the mean harder to reach, at the cost of low barriers that are easy to reach.Source: §5 · p. 9
  • Gemma relies mainly on a strong entropic bottleneck, Llama on a large energetic barrier, and Qwen on a mixture of both.Gemma used on-policy RL and Llama offline DPO; how alignment training maps onto the four model parameters is left for future work.Source: §5 · p. 9

6 CONCLUSION

The paper validates a physics-inspired microscopic model of BoN jailbreaking, finding that ASR does not follow a fixed power law and that the reported exponential crossover is a finite-dataset artifact. Its conclusions are bounded by the models, data, language, and automated judging considered.

6 CONCLUSION

  • The validated model extends BoN to multiple completions per augmentation and finds that ASR has no fixed power law; the reported exponential crossover is a finite-dataset artifact.The model also compares model safety and predicts defensive practices.Source: §6 · p. 9

ETHICS STATEMENT

  • The study covers intermediate-size open-weight models, one dataset, and English; several prompts in the safest models were censored, and judge disagreement remains possible.The reported µα results are accurate only up to a few tenths, corresponding to their error bars.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • A scripted pipeline produces the paper’s analyses and figures, and the authors say they will release the code and fitted per-prompt parameters.Source: REPRODUCIBILITY STATEMENT · p. 10

USE OF LARGE LANGUAGE MODELS

  • An AI coding assistant helped with analysis code, experiments, exploratory checks, supplementary drafting, and proofreading; the authors verified its outputs and take responsibility.The paper also uses LLMs as the study subject and an LLM-based jailbreak judge.Source: USE OF LARGE LANGUAGE MODELS · p. 10

A SADDLE-POINT EVALUATION OF THE POPULATION AVERAGE

The appendix evaluates the population-average expression by a saddle-point approximation, reducing its dependence on the attack budgets to a Lambert-W argument. It also describes how the collapse variables are obtained from measured prompt-level data.

A SADDLE-POINT EVALUATION OF THE POPULATION AVERAGE

  • The saddle-point condition yields a rescaled saddle ζ = W0(γ^2z), with z = (NM)e^−µE, reducing the scaling function to a function of that argument.The Gaussian fluctuations contribute a curvature factor φ′′(ξ⋆)^−1/2.Source: A SADDLE-POINT EVALUATION OF THE… · p. 13
  • For the Fig. 6(b) collapse, µR(M) is measured from prompt-level Rb(M), while the width is held at σE because measured σR(M) barely changes with M.For Qwen2.5-3B-Instruct, σR(M) changes from 0.55 at M = 1 to 0.54 at M = 64.Source: A SADDLE-POINT EVALUATION OF THE… · p. 13

B COMPUTE-OPTIMAL COMPLETION BUDGET

The compute-optimal split between augmentations and completions depends on their relative costs and the within-prompt spread of escape probabilities. The estimator details use a Beta soft wall, with limits on which prompts and completion budgets permit reliable fitting.

B COMPUTE-OPTIMAL COMPLETION BUDGET

  • If augmentations cost as much as or more than completions, M ⋆= 1 is optimal; spend the budget on distinct augmentations.This follows from the non-increasing Rb(M)/M relationship.Source: B COMPUTE-OPTIMAL COMPLETION BUDGET · p. 13
  • When prefill costs can be amortized, repeated completions are worthwhile only if within-prompt escape-probability spread outweighs that cost.For Qwen2.5-3B-Instruct, the reported r⋆ is ≃0.05; at r = 0.1, 0.4, 1, 2, 4, the fitted M ⋆ values are 1.3, 2.6, 4.3, 6.2, 9.1.Source: B COMPUTE-OPTIMAL COMPLETION BUDGET · pp. 13–14

C ESTIMATOR DETAILS AND IDENTIFIABILITY

  • The population survival estimate is counted directly over 400 prompts from 10^4-augmentation arms, without fitting or censoring.For Qwen2.5-3B-Instruct, every prompt was jailbroken at least 90 times.Source: C ESTIMATOR DETAILS AND IDENTIFIABILITY · p. 14
  • The hard support-edge estimator is too noisy, so the method fits a Beta soft wall to the Beta-Binomial completion counts and recovers αb from the fit.Synthetic-data checks found the shape parameters stayed within a few tenths of the ground truth, within reported error bars.Source: C ESTIMATOR DETAILS AND IDENTIFIABILITY · p. 14
  • The fit drops prompts with no jailbreaks and separately identifies its two parameters only when M ≳10 resolves the escape-probability distribution’s shape.The retained prompt counts vary by model, from 400 of 400 for Qwen2.5-3B-Instruct to 80 of 100 for Llama-3.1-8B-Instruct.Source: C ESTIMATOR DETAILS AND IDENTIFIABILITY · p. 14

D EXPERIMENTAL PROTOCOL

The experiments use AdvBench prompts and judge-scored completions, with generation settings, residual probes, and augmentation-scale calibration specified. The scale is calibrated separately for each model because embedding scales differ.

D EXPERIMENTAL PROTOCOL

  • Unsafe completions are those whose Granite-Guardian-3.0-2B harm probability exceeds τ = 0.5; τ values 0.1 and 0.3 are also recorded for threshold sweeps.HarmBench-Llama-2-13b-cls serves as a control on the two large grids.Source: D EXPERIMENTAL PROTOCOL · p. 14
  • Residual-stream experiments use Qwen2.5-3B-Instruct’s layer-30 last-token residual; the safety and saddle probes are ridge regressions with specified targets and penalties.The safety probe uses mean-pooled completion residuals and the judge’s binary label; the saddle probe predicts within-prompt-centred log p_b,a from augmented-prompt residuals.Source: D EXPERIMENTAL PROTOCOL · p. 14
  • Generation runs use the embedding or character kernel, with T = 1 unless stated otherwise, and cap completions at 96 new tokens.The character kernel flips case, permutes longer-word interiors, and replaces or duplicates characters at probabilities set by σ.Source: D EXPERIMENTAL PROTOCOL · p. 14, Table 2 · p. 15
  • σ is calibrated per model in raw embedding units to the first value where median per-prompt jailbreak rate enters the 1%–5% band.Embedding scales differ by an order of magnitude across model families, and the usable σ window can be narrow.Source: D EXPERIMENTAL PROTOCOL · p. 15

E AUGMENTATION SCALE ROBUSTNESS

The fitted four-parameter family varies with augmentation scale σ. The authors recommend matching median jailbreak rates, rather than fixing σ, when comparing models.

E AUGMENTATION SCALE ROBUSTNESS

  • The σ sweep uses Qwen-3B-Instruct with 100 behaviors, 50 augmentations, and 100 completions at each of six scales, holding prompts identical.The table reports the four parameters and effective barrier for Qwen2.5-3B-Instruct at 100 × 50 × 100 per σ, with the main text using σ = 0.048.Source: E AUGMENTATION SCALE ROBUSTNESS · p. 15, Table 3 · p. 15
  • Higher σ progressively lowers μ_α and raises μ_β, consistent with erosion of the model’s safety barriers.At sufficiently high σ, noise destroys too much of the original prompt and completions lose coherence.Source: E AUGMENTATION SCALE ROBUSTNESS · p. 15
  • For cross-model comparisons, match median jailbreak rates rather than holding σ fixed, because the parameter trends follow that rate as a proxy for attack strength.The main-text grids use one calibrated σ per model.Source: E AUGMENTATION SCALE ROBUSTNESS · p. 15

F AUGMENTATION KERNEL ROBUSTNESS

The model’s Gaussian and log-normal structure persists across augmentation kernels, although fitted safety parameters depend on attack type. At approximately equal median rates, the fitted families largely agree within noise.

F AUGMENTATION KERNEL ROBUSTNESS

  • The four safety parameters depend on attack type, while the model’s entropic-bottleneck, hyperplane, and exponential-saddle structure is described as universal.The fits can compare models under the same augmentation or a single model under different augmentation types.Source: F AUGMENTATION KERNEL ROBUSTNESS · p. 16
  • At approximately equal median rate, character-kernel σ = 0.25 and embedding-kernel σ = 0.030 yield four-parameter families that largely agree within noise.Table 4 compares the two kernels on the same model and 100 prompts.Source: F AUGMENTATION KERNEL ROBUSTNESS · p. 16, Table 4 · p. 16
  • Across the character kernel, standardized α_b remains indistinguishable from Gaussian and standardized log β_b from normal; reconstruction of ASR@N,M for N ≤ 100 and M ≤ 50 has mean absolute error 0.019.The authors conclude that the four-parameter structure is universal across augmentation kernels.Source: F AUGMENTATION KERNEL ROBUSTNESS · p. 16

G JUDGE ROBUSTNESS

Changing the judge changes unsafe labels and fitted barrier scales, but the Gaussian and log-normal distribution families persist. The stricter judge also widens both fitted spreads.

G JUDGE ROBUSTNESS

  • On the identical Qwen2.5-3B-Instruct grid (400 prompts × 100 augmentations × 300 completions), HarmBench-Llama-2-13b-cls labels 0.3% unsafe versus Granite-Guardian-3.0-2B’s 4.4%.No generation differs between the judges; only the label does.Source: G JUDGE ROBUSTNESS · p. 16
  • Under both judges, α_b remains indistinguishable from Gaussian and β_b stays log-normal, despite HarmBench’s stricter labeling.The table caption also reports that the ordering of the two channels is unchanged.Source: G JUDGE ROBUSTNESS · p. 16, Table 5 · p. 16
  • The stricter judge widens σα from 0.65 to 1.04 and σβ from 0.52 to 0.77, pushing the hardest prompts further into the tail.All fit parameters depend on the judge’s notion of safety.Source: G JUDGE ROBUSTNESS · p. 16
Loading 2609.32116v1…