PaperlayerView exact revision on arXiv

Source-linked AI summary

ECHO: Replacing the Compliance Proxy with Response Harmfulness in Optimization-Based Jailbreak Attack

Yuanwei Wu, Lingdong Huang, Yunlong Mao, Kanye Ye Wang, Jingyu Hua, Sheng Zhong

arXiv:2610.23030v1cs.CRcs.CL

TL;DR

Optimization-based jailbreaks maximize token-level compliance proxies rather than evaluating the model’s decoded response. ECHO replaces that proxy with HarmSig response-harmfulness scoring and combines it with word-level perturbations and Keyword-Anchored Beam Search; across five open-weight victims, it ranks first under StrongREJECT and HarmBench.

  • Problem

    Optimization-based jailbreaks use prompt-level compliance proxies that cannot distinguish cautious hedges from fully actionable harmful responses.

    Source: §1 · p. 2

  • Method

    ECHO uses HarmSig, a response-level harmfulness scorer, with five word-level perturbation operators and Keyword-Anchored Beam Search under a black-box threat model.

    Source: §1 · p. 2

  • Results

    75.19–89.23% HarmBench success; ECHO ranks first under both StrongREJECT and HarmBench on every victim, while no baseline exceeds 65%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ECHO-generated prompts transfer across model families without further search, indicating portable adversarial structure.

    Source: §5 · p. 9

    Independent safety classifiers in commercial systems can further reduce attack success, and ECHO’s large query budget makes direct API optimization costly.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Despite extensive safety alignment, large language models remain vulnerable to jailbreak attacks. Existing optimization-based attacks share a single objective: maximize a token-level proxy of victim compliance---such as the likelihood of an affirmative prefix---computed on the prompt rather than the decoded reply. This proxy cannot distinguish a cautious hedge from a fully actionable harmful answer, leaving the search blind to what the model actually says. We propose ECHO (Evaluation-Conditioned Harmfulness Optimization), which replaces the compliance proxy with response-level, continuous harmfulness scores from HarmSig, a 1.7B regressor grounded in actual harmful capability. ECHO couples HarmSig with five word-level perturbation operators and Keyword-Anchored Beam Search with global top-m memory, under a strict black-box threat model. Across five open-weight victims on AdvBench, ECHO achieves up to 0.898 StrongREJECT mean score and 89.23% HarmBench success. Our code is publicly available at https://github.com/STANWYW/echo.

1 Introduction

Optimization-based jailbreaks typically optimize token-level compliance proxies rather than the harmfulness of the decoded reply, so they cannot distinguish cautious hedges from actionable answers. ECHO instead optimizes response-level harmfulness, and its reported AdvBench results rank first across five open-weight victims.

1 Introduction

  • Existing optimization-based attacks optimize affirmative-prefix likelihood rather than whether the decoded response delivers harmful content.A cautious hedge and an actionable harmful answer can receive the same proxy score.Source: §1 · pp. 1–2
  • A suitable response scorer must be continuous, fast enough for thousands of search branches, and grounded in actual harmful capability.Binary metrics lack continuous ranking; API rubrics and large-model classifiers lack the required speed.Source: §1 · p. 2
  • ECHO combines HarmSig, five word-level perturbation operators, and Keyword-Anchored Beam Search with global top-m memory under a strict black-box threat model.HarmSig is a 1.7B regressor distilled from three frontier teacher judges on 29,744 jailbreak-trajectory responses.Source: §1 · p. 2
  • 75.19–89.23% HarmBench success; ECHO ranks first under both StrongREJECT and HarmBench on every victim, while no baseline exceeds 65%.Evaluations cover AdvBench, five open-weight victims, and six HarmBench harm categories.Source: §1 · p. 2

2 Related Work

Related jailbreak work includes fixed handcrafted transformations and attacker-LLM-generated prompts, while optimization-based methods iteratively perturb prompts using compliance proxies. ECHO differs from those optimization objectives by ranking prompts using harmfulness of the victim’s full decoded reply.

2.1 Non-Optimization Jailbreak Attack

  • Handcrafted jailbreaks use fixed or rule-based transformations, so their diversity is bounded by human creativity and identified surfaces can be patched.Examples include text flipping, ASCII encoding, multilingual interactions, role framing, and restructuring dialogues into single-turn prompts.Source: §2.1 · p. 2
  • LLM-generated jailbreaks use an attacker model for prompt refinement, paying its inference cost at each iteration and potentially facing its safety alignment.Methods include social-engineering or tree-search loops, embedding-space Bayesian optimization, multi-agent reflection, reinforcement learning, and trained generators.Source: §2.1 · p. 2

2.2 Optimization-Based Jailbreak Attack

  • Optimization-based jailbreaks vary in search machinery, but commonly optimize token-level proxies of victim compliance rather than harmfulness of the full reply.Prior remedies such as refusal-token suppression and per-model prefix targets remain within the compliance-matching family; ECHO instead scores the decoded reply with HarmSig.Source: §2.2 · p. 3

3 Methodology

ECHO searches for reachable prompts whose decoded replies receive high HarmSig harmfulness scores, under a black-box threat model. Its methodology combines keyword-preserving word edits, beam search, response scoring, and operator-weight calibration.

3.1 Problem Setup

  • ECHO assumes single-turn black-box access and selects reachable prompts by response harmfulness, excluding gradients, logits, attacker LLMs, templates, and fixed affirmative targets.The objective uses HarmSig rather than a target string for the victim’s reply.Source: §3.1 · p. 3

3.2 Framework Overview

  • Offline preparation trains HarmSig on real jailbreak-trajectory responses and calibrates operator weights; online search edits prompts, filters for keyword feasibility, and scores full replies.The online loop terminates when a candidate reaches threshold α or the step budget is exhausted.Source: §3.2 · p. 4

3.3 Perturbation Operators

  • CORE and PERIPH replace core or peripheral words with synonyms, while a fixed keyword vocabulary preserves the malicious goal’s semantic identity.The vocabulary includes non-stopword content words from the goal and their WordNet synonyms.Source: §3.3 · p. 4
  • PRESSURE requests greater detail, DELETE removes a trigger or softener only if keywords remain feasible, and SWAP transposes adjacent non-protected words.The mixture weights determine which edit types are sampled along search trajectories.Source: §3.3 · p. 4

3.4 Keyword-Anchored Beam Search

  • At each depth, keyword-feasible children are scored and the top k advance, while a global top-m memory retains strong candidates that beam pruning might discard.Search stops at threshold α or after at most T depths; cached responses allow best-of-m reporting without extra victim queries.Source: §3.4 · pp. 4–5

3.5 HarmSig: The Inner-Loop Evaluator

  • HarmSig is a continuous response scorer trained by MSE on 29,744 teacher-labeled responses and applied in one pass; a control replaces it with GCG’s prefix-likelihood proxy.The labels are the mean scores of three frontier teachers, preserving a continuous regression target.Source: §3.5 · p. 5

3.6 Bayesian Optimization over Operator Weights

  • Bayesian optimization tunes the operator mixture to maximize, over a validation pool, each goal’s best HarmSig score among prompts visited during search.The mixture lies on the simplex and is fixed before large-scale experiments.Source: §3.6 · p. 5

4 Experiments

ECHO is evaluated on AdvBench and HarmBench across five open-weight victims, with StrongREJECT and HarmBench success rate used to compare results. Experiments report strong benchmark performance, cross-model transfer, scorer and search ablations, and runtime and human-validation checks.

4.1 Experimental Setup

  • The evaluation uses 520 AdvBench behaviors and the standard 200-behavior HarmBench subset, across five instruction-tuned open-weight victims; ECHO starts from raw goals without jailbreak templates.The reported configuration uses beam width k=10, branch factor b=5, horizon T=20, early-stop threshold α=0.8, and memory size m=25; top-m=25 uses cached responses and requires no additional victim queries.Source: §4.1 · pp. 5–6, §4.2 · p. 6, Table 1 · p. 6

4.2 Main Results

  • Across all five victims, ECHO (top-m=25) is strongest under both metrics; Qwen2.5-7B reaches 0.898 / 89.23% and Gemma-3-12B-IT reaches 0.702 / 81.54%.Both metrics saturate near m≈20–25, and their model rankings agree on four of five victims.Source: §4.2 · pp. 6–7, Figure 4 · p. 6

4.3 Cross-Model Transfer

  • ECHO prompts transfer across victims: Gemma-3-12B-IT as source reaches 0.786 / 65.00% on Qwen2.5-7B and 0.741 / 60.77% on Llama-3.1-8B-Instruct.No off-diagonal entry is near zero; Qwen2.5-7B transfers more weakly to the Llama-2 family (0.436–0.458 SR) than to Llama-3.1-8B (0.690 SR).Source: §4.3 · p. 7, Table 2 · p. 7

4.4 Scorer Ablation

  • With search procedure, operator pool, victim, budget, and evaluation held fixed, HarmSig ranks above the compliance proxy, which reaches only 0.253 SR / 8% HB.The reported ordering is Random < Keyword < Compliance proxy < HarmSig; the proxy elicits affirmative openings but does not steer replies toward harmful content.Source: §4.4 · pp. 7–8, Table 3 · p. 7

4.5 Search Ablation

  • Adding keyword-feasibility, Bayesian-optimized operator weights, and global top-m memory raises mean SR by +0.167 and HB by +11 points over plain beam (0.626 / 48% → 0.793 / 59%).Beam search substantially outperforms Greedy and simulated annealing; global memory rescues high-scoring candidates pruned at earlier depths.Source: §4.5 · p. 8, Table 4 · p. 8

4.6 Additional Experiments

  • On Llama-2-7B-Chat, ECHO takes 81.80 seconds per goal, faster than GCG and AutoDAN but slower than DSN.Early stopping reduces victim queries from at most 1000 to an average of 877 per goal; blinded validation on 200 responses yielded 95.0% annotator agreement and Cohen’s κ=0.88.Source: §4.6 · pp. 8–9, Table 5 · p. 9, Table 6 · p. 9

5 Conclusion

ECHO replaces token-level compliance with HarmSig response-harmfulness scoring in a black-box jailbreak framework. The conclusion reports stronger results than the compliance proxy and transfer across model families without further search.

5 Conclusion

  • 0.793 mean SR and 59% HarmBench for HarmSig compare with 0.253 mean SR and 8% HarmBench for the white-box compliance proxy.The scorer-swap ablation reports these results even though HarmSig operates under fully black-box conditions.Source: §5 · p. 9
  • ECHO combines HarmSig, five word-level perturbation operators, and Keyword-Anchored Beam Search to outperform every baseline on AdvBench across five open-weight victims.The conclusion attributes the search objective to response-level harmfulness rather than a token-level compliance proxy.Source: §5 · p. 9
  • ECHO-generated prompts transfer across model families without further search, which the authors describe as evidence of portable adversarial structure.The conclusion also says current safety alignment remains brittle to surface-level edits when search attends to the decoded response rather than opening tokens.Source: §5 · p. 9

Limitations

The authors identify closed-source deployment as a boundary: commercial systems may add safety classifiers, and direct optimization against APIs is costly.

Limitations

  • Independent input/output safety classifiers in commercial systems can further reduce ECHO's attack success.The authors identify these unobserved safety objectives as a limitation for closed-source deployment.Source: Limitations · p. 9
  • ECHO's large query budget makes direct optimization against commercial APIs costly.Zero-query transfer avoids that optimization cost, but transfer to GPT-5.4 reaches only 10.0% HarmBench success under top-m=25.Source: Limitations · p. 9
  • The multilingual evaluation covers only French and German.The authors say extending ECHO to Chinese and Japanese remains challenging because its word-level operators and keyword constraints rely on reliable segmentation and lexical resources.Source: Limitations · p. 9

Ethical Considerations

The authors frame ECHO as safety research while acknowledging possible misuse and describing defensive motivations and safeguards. Their threat-model comparison evaluates six attacker-resource conditions.

Ethical Considerations

  • The authors identify a dual-use risk: ECHO could lower the technical barrier to generating effective jailbreak prompts.They say experiments were conducted in a controlled setting and commercial-model evaluation was limited to zero-query transfer without target-side optimization.Source: Ethical Considerations · p. 9
  • The authors cite vulnerability discovery and the resulting prompts' value as training data for alignment and content filters as reasons for the work.They state that the research is intended to help identify weaknesses, create adversarial training datasets, and inform more robust alignment.Source: Ethical Considerations · pp. 9–10
  • The authors argue that ECHO's black-box methodology reflects realistic threat models defenders must consider.They encourage responsible disclosure and defensive applications.Source: Ethical Considerations · p. 10

A Threat-Model Comparison

  • Table 8 compares 21 methods across six conditions, including victim access, auxiliary attacker LLM use, prompt scaffolding, target strings, and interaction turns.A checkmark indicates a method satisfies a condition; a cross indicates it requires the resource.Source: A Threat-Model Comparison · p. 12

B HarmSig Student Training

HarmSig is trained on teacher-scored jailbreak responses, with held-out validation used to select a checkpoint. The appendix reports its validation agreement and the point at which top-m results saturate.

B HarmSig Student Training

  • HarmSig training uses 25,284 examples after reserving 4,460 of 29,744 consensus-labeled responses for validation.Goals are filtered against AdvBench and HarmBench goals using a character-level 5-gram Jaccard similarity threshold of 0.88; each response is truncated to 1,024 subwords.Source: B HarmSig Student Training · p. 12
  • Table 8 compares optimization-based jailbreak methods across six threat-model conditions, marking whether each method satisfies a condition or requires its resource.The conditions include gradient and logit access, auxiliary attacker LLM use, predefined scaffolding, a fixed affirmative target, and single-turn operation.Source: Table 8 · p. 13
  • Training MSE falls from 0.021 after epoch 1 to 0.0058 after epoch 3, alongside improving validation metrics.Table 9 summarizes student performance across three epochs and uses 4,460 held-out responses for validation.Source: B HarmSig Student Training · p. 13, Table 9 · p. 13
  • The best configuration reaches Spearman ρ=0.846, RMSE 0.164, MAE 0.105, accuracy 0.896, and F1 0.820 on held-out teacher scores.Accuracy and F1 use a 0.5 score threshold; precision is 0.834 and recall is 0.807.Source: B HarmSig Student Training · p. 13

C Per-Model Top-m Saturation Values

  • For top-m saturation, reported values remain unchanged beyond m=100; unrounded HarmBench values are constant there, while remaining StrongREJECT changes are below 0.001 and end by m=105.Table 10 samples m=1 and then every five candidates from m=5 through m=100.Source: C Per-Model Top-m Saturation Values · p. 13

D HarmBench Results by Semantic Category

Across five victims, Cybercrime and Chemical & Biological are the most vulnerable HarmBench categories, while Harassment is consistently hardest. Misinformation shows the widest StrongREJECT–HarmBench divergence, reflecting different assessments of hedged, low-actionability responses.

D HarmBench Results by Semantic Category

E Compute Infrastructure

F Multilingual Robustness

ECHO remains effective on French and German AdvBench goals without WordNet-based perturbations. A trajectory-guided search strategy lowers victim-query use while largely preserving attack effectiveness.

F Multilingual Robustness

  • For French, ECHO achieves SR/HB of 0.806/92% with top-m=25; for German, it achieves 0.884/97%.With top-m=1, the corresponding French and German SR/HB scores are 0.627/70% and 0.661/85%.Source: F Multilingual Robustness · p. 14

G Trajectory-Guided Query Reduction

  • Successful trajectories outside the 100-goal evaluation set exhibit recurring operator transitions, notably SWAP→DELETE and SWAP→PRESSURE.Source: G Trajectory-Guided Query Reduction · p. 14
  • The trajectory-guided strategy progressively reduces beam and branch sizes when HarmSig indicates a promising trajectory, applying conservative pruning only later in search.Source: G Trajectory-Guided Query Reduction · p. 15
  • On the same 100 AdvBench goals, trajectory-guided selection and pruning reduce average victim queries from 871.50 to 779.28 per goal, a 10.58% reduction.Average runtime falls from 81.80s to 70.06s per goal; paired confidence intervals for quality differences include zero.Source: G Trajectory-Guided Query Reduction · p. 15

H Closed-Source Model Experiment

ECHO prompts ranked on Llama-2-7B-Chat were transferred to GPT-5.4 without target-side search or refinement. They achieved a StrongREJECT score of 0.564 and a HarmBench success rate of 10.0% with top-m=25.

H Closed-Source Model Experiment

  • For each of 100 AdvBench goals, the experiment freezes HarmSig-ranked prompts from Llama-2-7B-Chat and evaluates them on GPT-5.4 without target-side search or refinement.Source: H Closed-Source Model Experiment · p. 15
  • Transferred prompts achieve an SR score of 0.564 and a HarmBench success rate of 10.0% under top-m=25, demonstrating measurable transfer to a closed-source target without additional optimization.Source: H Closed-Source Model Experiment · p. 15

Related papers

Loading 2610.23030v1…