PaperlayerView exact revision on arXiv

Source-linked AI summary

Refusing Everything Looks Safe: Restoring the Benign Arm to Encoded-Prompt Evaluation

Haoyu Zhang, Haowen Xu, Xiao Luo, Hanwen Liu, Yang Chen, Zijian Xiao, Yi Feng, Xiangchen Guan, Mohammad Zandsalimy, Shanu Sushmita

arXiv:2609.26176v2cs.CRcs.AI

TL;DR

Encoded-prompt evaluations mostly test harmful requests, leaving benign content in attack form unmeasured. This paper compares harmful and benign arms across plaintext, scaffold, and encoded conditions; it finds little cross-model separation on the harmful arm and much wider variation on the benign arm.

  • Problem

    Encoded-prompt benchmarks generally send harmful encoded requests and report compliance or refusal, while omitting benign inputs in the same format.

    Source: §1 · p. 1

  • Method

    The paper evaluates harmful and theme-matched benign requests in plaintext, scaffold, and encoded conditions, using the scaffold to separate template effects from character transformation.

    Source: §3 · p. 3

  • Results

    0.08: harmful-arm spread across four models under homoglyph encoding, versus 0.57 in plaintext; a full pipeline’s +0.26 harm-gap rise is unresolved by the standard harmful-arm metric.

    Source: §1 · pp. 1–2, §9 · p. 7

  • Takeaways & Limitations

    Encoded-attack evaluations should report both harmful and benign arms; the harm gap requires the benign arm and a plaintext baseline.

    Source: §9 · p. 7

    The cross-model result uses homoglyph encoding, and its echo screen correlates with harm-gap magnitude, discarding larger encoded gaps somewhat more often.

    Source: §8 · p. 7

Abstract

from arXiv · show

Encoded-prompt attacks are evaluated almost entirely on their harmful arm: a benchmark sends obfuscated harmful requests and reports how often the model complied. A high refusal rate there is reported as safety, and it is equally consistent with a model that has stopped telling the request apart from anything else in the same format. We run the benign arm through the same transformation, and the two cases are far apart. Across four 7-8B models spanning three base families and four post-training recipes, refusal of harmful homoglyph-encoded prompts spans 0.08 while the same four span 0.57 on the identical requests in plaintext. What the encoding destroys is not refusal but the harm gap: on one model the gap between harmful and benign refusal falls from +0.82 in plaintext to exactly 0.00 under the encoding, and a benchmark reading only the harmful arm scores that model and one retaining a +0.61 gap identically. Running the cell such benchmarks leave out (plaintext content wearing the attack template, with nothing obfuscated) shows that on two of the four models the loss is caused by the protocol rather than by the character transformation, and on a third by the characters. Across a full SFT -> DPO -> RLVR pipeline the harm gap rises by +0.26 with a paired interval excluding zero while the standard harmful-arm metric registers no resolved change at all. We report twelve instrument defects, each with the control that caught it, including a binary jailbreak judge that fires on 0.61-0.70 of responses to plaintext benign prompts; six of the twelve inflate apparent safety, which is the direction a broken safety evaluation fails in by default.

1 Introduction

Encoded-prompt benchmarks generally test harmful requests while omitting benign inputs in the same format. The paper argues that this leaves the harmful-arm refusal rate unable to distinguish models that track harm from models that refuse by appearance.

1 Introduction

  • Encoded-prompt attacks are commonly evaluated on harmful requests alone, leaving the benign half of the input space unmeasured.The cited protocol reports how often the model complies or refuses after harmful requests are encoded.Source: §1 · p. 1
  • 0.08: harmful homoglyph-refusal spread across four models, versus 0.57 for the same requests in plaintext, leaves the harmful arm with little ranking power.The encoded spread is at the edge of what sampling noise alone would produce if the four models were identical.Source: §1 · p. 1, Figure 1 · p. 2
  • The harm gap, harmful minus benign refusal, can distinguish a +0.61 gap from a 0.00 gap that a harmful-only metric scores identically.The authors frame this as a measurement claim, not a claim that refusing encoded text is unsafe.Source: §1 · pp. 1–2
  • The paper’s four-step argument tests harmful-arm resolution, separates template from character effects, checks a full pipeline, and audits instrument defects.Six of twelve reported defects inflate apparent safety.Source: §1 · p. 2
  • The cross-model finding is limited to homoglyph substitution on four open-weight 7–8B models; the pipeline result covers one family’s published recipe.The authors do not establish why refusal stops tracking harm.Source: §1 · p. 2

2 Related work

Related work studies encoded attacks, harmful-arm scoring, and over-refusal, but the paper identifies benign content in attack form as a cell those evaluation lines leave unmeasured.

2 Related work

  • Encoded prompts probe whether safety training generalizes beyond familiar surface forms; prior work reports cipher attacks bypassing alignment, which this paper does not dispute.Wei, Haghtalab, and Steinhardt (2023) name this mismatch; Yuan et al. (2024) report cipher-encoded chat bypassing alignment on frontier models.Source: §2 · pp. 2–3
  • Prior work improves harmful-arm scoring, while this paper argues that even a better judge cannot recover model separation when the harmful arm carries little information.The paper also reproduces a binary-judge failure on plaintext benign prompts.Source: §2 · p. 3
  • XSTest measures over-refusal, while JailbreakBench supplies theme-matched benign prompts; neither over-refusal nor encoded-attack evaluation covers benign content in attack form.The authors adjudicated eleven transformation-attack papers and two over-refusal benchmarks; neither XSTest nor OR-Bench contains encoding or obfuscation terms.Source: §2 · p. 3
  • The same missing benign-under-encoding measurement applies to production prompt guards, whose false-positive rates on plain prompts do not measure the deployed condition.The paper notes this as a boundary on what its target-model result says about deployed systems.Source: §2 · p. 3
  • The authors attempted a probe-based account of the harm gap but withdrew it because the attempt failed.They place this effort alongside interpretability work locating refusal in model internals.Source: §2 · p. 3

3 Method

The method compares matched harmful and benign requests across plaintext, scaffold, and encoded conditions. Its scaffold isolates the attack template from character transformation, while the harm gap measures the difference in refusal between the two arms.

3 Method

  • Four open-weight instruction-tuned models from independently developed post-training pipelines are evaluated on JailbreakBench harmful prompts and theme-matched benign counterparts.Each cell contains 100 requests.Source: §3 · p. 3
  • The six-cell design crosses harmful and benign content with plaintext, scaffold, and encoded conditions; the scaffold wraps untransformed content in the attack template.Comparing plaintext with encoded alone attributes the template’s effect to encoding; the scaffold separates the two at the cost of one forward pass.Source: §3 · p. 3
  • The harm gap is the difference between harmful and benign refusal rates within a condition; gap lost compares that gap in plaintext with the encoded condition.Neither quantity can be computed from an attack benchmark without the benign arm and plaintext baseline.Source: §3 · pp. 3–4
  • Refusal is classified by an LLM judge validated against two cross-family annotators, and rerunning a condition shifts the harm gap by 0.01–0.06.The judge and run-to-run variation are detailed in the supplementary material.Source: §3 · p. 4
  • Cross-model spreads are compared with a bootstrap null for four identical models at n=100, rather than interpreted directly as stable summaries.For paired encoded verdicts, the null preserves shared item difficulty; plaintext spreads use an independent null because per-item verdicts were not retained.Source: §3 · p. 4, Table 1 · p. 4

4 The harmful arm cannot separate models

On homoglyph-encoded harmful prompts, the four models’ refusal rates are compressed to a spread of 0.08, while their benign refusal rates span 0.69. The harmful arm therefore misses distinctions reflected in the harm gap, including a 0.00 versus +0.61 gap between two models.

4 The harmful arm cannot separate models

  • 0.57 is the plaintext harmful-refusal spread, against a null of 0.08 [0.02, 0.16].The homoglyph spread remains marginally separable, so the result is compression rather than equality.Source: §4 · p. 4
  • 0.08 is the homoglyph harmful-refusal spread across four models, compressed to the edge of what n=100 resolves rather than showing equality.The paired null places the 95th percentile at 0.07; the observed spread is marginally separable (p = 0.03).Source: §4 · p. 4
  • 0.69 is the homoglyph benign-refusal spread, against a paired null of 0.08 [0.02, 0.16].The observed spread exceeds the 99th percentile under both paired and independent nulls (p < 10−4 either way).Source: §4 · p. 4
  • 0.00 versus +0.61 harm gaps distinguish Llama-3.1-8B and Qwen2.5-7B despite only a 0.08 harmful-arm difference.The harmful difference is real, but 8× smaller than the harm-gap difference.Source: §4 · p. 4
  • Three of four models show significant harm-gap collapse; Mistral’s interval includes zero.The difference-of-differences intervals are Llama +0.82 [+0.74, +0.90], Tülu-3 +0.34 [+0.20, +0.48], Qwen +0.20 [+0.07, +0.33], and Mistral +0.09 [−0.06, +0.24].Source: §4 · p. 4
  • Two of seven encoding rungs survive both readability and echo-cleanliness screens on more than one model.Homoglyph is readable on all four models; math-bold is readable on three and clears the echo screen on all four.Source: §4 · pp. 4–5
  • Mistral’s math-bold decode ability is 0.00 because it refuses both arms when it cannot read them; no harm gap is defined.This is a decode failure, not a failure to distinguish harmful from benign content.Source: Table 2 · p. 5
  • Math-bold does not replicate homoglyph: their Llama harm gaps are 0.00 and +0.18, respectively.On the other three models the two rungs agree in sign and rank models identically; math-bold produces the largest measured harm gap, +0.77 on Qwen.Source: §4 · p. 5

5 Splitting the protocol: template or characters

The authors add a plaintext-content condition under the attack template to separate template effects from character-transformation effects. The template dominates the harm-gap loss on Llama and Tülu-3, while characters dominate on Qwen; the reported intervals are unpaired, and one scaffold echo rate is left uncorrected.

5 Splitting the protocol: template or characters

  • On Llama, the template accounts for +0.67 [+0.56, +0.78] of a total harm-gap loss of +0.84 [+0.76, +0.92].On Tülu-3, the template accounts for +0.28 [+0.15, +0.41] of +0.37 [+0.23, +0.51], while its character term is not separable from zero.Source: §5 · p. 5
  • On Qwen, the character term is +0.25 [+0.11, +0.39], while the template term is +0.02 [−0.09, +0.13].On Mistral, the two terms have opposite signs and partly cancel; its total includes zero while its character term does not.Source: §5 · p. 5
  • The reported intervals are unpaired because encoded-arm per-item verdicts are the only ones persisted; their widths are upper bounds.The scaffold condition leaves content untransformed, making Mistral’s 0.38 echo rate indistinguishable from genuine parroting, so it is reported without correction.Source: §5 · p. 5

6 A published safety pipeline does not repair it

The Tülu 3 pipeline improves harm gaps in plaintext and under homoglyph encoding by nearly the same amount, leaving their shortfall unchanged. A harmful-arm-only metric does not resolve the encoded-gap improvement, and the template control shows that some loss can precede character transformation.

6 A published safety pipeline does not repair it

  • The paired comparison uses identical base weights and measures plaintext and encoded arms at each of the three Tülu 3 post-training stages within one job.The cross-model comparison alone cannot isolate what produces differences because models vary in data, objective, and scale.Source: §6 · p. 5
  • On Llama, the scaffold gap falls from +0.83 to +0.16 before any characters are transformed, entirely through benign refusal rising from 0.10 to 0.83.On Qwen, the scaffold has no such cost.Source: Table 3 · p. 6
  • +0.26: the encoded harm gap rises from +0.20 to +0.46, with a paired 95% CI of [+0.14, +0.38], while the plaintext gap improves by +0.25.The shortfall between the gaps remains 0.35 → 0.34; the result visual reports these changes for the Tülu 3 pipeline.Source: §6 · p. 5, Table 4 · p. 6
  • The earlier analysis lacked a plaintext arm, and the required echo screen rejects four of nine cells in the table analysis.For two other encodings, harmful-arm refusal falls, but those echo-contaminated cells are not reportable; the direction is presented rather than as a measured effect.Source: §6 · p. 6
  • Harmful-arm refusal under homoglyph encoding moves 0.99 → 0.94, but the change does not resolve (exact McNemar p = 0.06), unlike the harm-gap increase.Five of 100 items flip, all in the same direction.Source: §6 · p. 6

7 The instrument

The paper identifies twelve instrument defects and reports the controls that exposed them. Six inflate apparent safety, while a binary judge error runs in the opposite direction.

7 The instrument

  • Without a plaintext denominator, an encoded benign refusal rate of 0.99 is ambiguous: it is catastrophic against 0.10 and unremarkable against 0.95.The paper makes the plaintext arm model-level and mandatory.Source: §7 · p. 6
  • Without a benign arm, 0.99 harmful refusal on Llama-3.1-8B fits both harm recognition and refusal of anything in the encoded format.The benign arm distinguishes those hypotheses.Source: §7 · p. 6
  • On plaintext benign prompts, the binary jailbreak judge fires on 0.61–0.70 of responses, where any positive is judge error.On the harmful arm, its output is one minus refusal to within 0.01; the paper reports no attack-success number.Source: §7 · p. 6
  • The refusal judge scores a response that only echoes ciphertext as refusal, so each condition is screened using non-echoing cells and the harm-gap displacement.Homoglyph clears the screen on all four models; reported rates retain the full n=100 denominator.Source: §7 · p. 6
  • Defects (5)–(12) include length confounds, inadequate probe controls, binary axes, exact-match misses, item leakage, an unrun screen, missing benign-arm comprehension, and unpaired intervals.The paper reports each defect with its control; a re-run placed the two arms within 0.02 on fifteen of sixteen comprehension cells.Source: §7 · p. 7
  • Six defects inflate apparent safety; defect (3), a binary judge error, is the single exception and overstates attack success.Defects (5), (6), and (9) affect the probe axis, while (11) and (12) are absences of record rather than measurement errors.Source: §7 · p. 7

8 Limitations

The cross-model result uses homoglyph substitution, the only encoding that passes every screen on all four models; the pipeline adds two encodings. The echo screen is outcome-related, and several further limitations remain unresolved.

8 Limitations

  • Homoglyph is the only encoding passing every screen on all four models; the pipeline result adds two more encodings.The echo screen is a stability criterion on the reported gap, so it is not outcome-independent.Source: §8 · p. 7
  • Across 27 model-by-encoding cells, screen passage correlates with gap magnitude at r = −0.49 (p = 0.01), and with the driving echo rate at r = −0.59 (p = 0.001).Larger encoded gaps are discarded slightly more often; the authors disclose this direction rather than correcting for it because removing the screen would admit cells with majority-artifact refusal verdicts.Source: §8 · p. 7
  • Other limitations include run-to-run non-determinism at n=100, unresolved equalisation-versus-saturation and lost-versus-unused-harm-information readings, and a withdrawn inference.These are stated in full in the supplementary material.Source: §8 · p. 7

9 Conclusion

The conclusion argues that encoded-prompt evaluation omits the benign arm and plaintext baseline needed to assess whether refusal tracks request content or appearance. In the reported comparisons, the pipeline improves harm gaps similarly in plaintext and under encoding, leaving their shortfall unchanged.

9 Conclusion

  • On the harmful arm, four models converge to within 0.08, while their benign-arm refusal spans two-thirds of the range.The conclusion recommends reporting both arms in encoded-attack evaluations.Source: §9 · p. 7
  • Assessing whether refusal tracks the request or its appearance requires the benign arm and a plaintext baseline, neither of which is standard.Source: §9 · p. 7
  • A published safety pipeline improves the harm gap in plaintext and under encoding by nearly the same amount, so their shortfall is untouched and the standard metric registers none of it.Source: §9 · p. 7

Supplementary Document

The supplement provides reproducibility details, a paper-by-paper account of prior benign controls, interval reporting, and fuller descriptions of instrument defects. Its controls show that both refusal judging and interpretation of echoing responses can materially affect encoded-prompt evaluation.

A Reproduction details

  • The supplement records only 12–58% byte-identical responses across nominally greedy repeats, alongside 1.0–4.3% verdict flips.The largest repeated-run gap movement reported for the reproduction check is 0.010 on Tülu/fullwidth.Source: A Reproduction details · pp. 10–11

B.1 The thirteen rows, adjudicated

  • Prior-work adjudication distinguishes no benign arm (B0), plaintext-only benign controls (B1), transformed benign capability tests (B2), and transformed benign refusal rates (B3).B3 is the cell of interest; B2 is a near miss because it measures decoding or answering rather than refusal.Source: §B.1 · p. 11

B.2 The near misses, and the one paper that runs the cell

  • Wei, Haghtalab, and Steinhardt (2023) used one harmless capability-control prompt, while Handa et al. (2025) report a false-positive rate on a different corpus from CipherBench.The paper’s paired measurement uses the same items as benign and harmful, in plaintext and transformed form, to compute the harm gap.Source: §B.2 · p. 11

C Intervals on every reported rate

  • The supplement places intervals for the cross-model and three-arm rates in Table 7 and the post-training ladder intervals in Table 5.Table 7 uses Wilson intervals for rates and Wald intervals for harm gaps between different prompt sets.Source: C Intervals on every reported rate · p. 11

D.1 Defect (3): the binary jailbreak judge

  • The binary jailbreak judge flags 0.61–0.70 of plaintext benign responses, so its attack-success output is not reportable.On harmful prompts, its output is within 0.01 of one minus the refusal rate on all four models.Source: §D.1 · p. 11

D.2 Defect (4): echo scored as refusal, and two caveats

  • Bare ciphertext is classified as refusal in 100% of tested cases, prompting a screen that makes echo-related gap movement a reporting criterion.Homoglyph clears the screen on all four models; fullwidth and zero-width fail on three of four.Source: §D.2 · p. 12, Table 5 · p. 12
  • The encoded harmful refusal rates sit near ceiling at 0.91–0.99, so equalisation and saturation both fit the observed cells at n=100.The supplement identifies this as an unresolved interpretation rather than claiming which account is correct.Source: §D.2 · p. 12

D.3 Defects (5) to (9), and (12)

The instrument audit identifies confounds and recording failures in length, probe, and pipeline analyses, then reports corrections and validation. It also documents run-to-run variability and the limits of the available item-level records.

D.3 Defects (5) to (9), and (12)

  • The harmful and benign corpora differ in raw character length (AUROC 0.654), but length matching moves the harm gap by at most 0.024.Token-based checks also leave the stratified gap within 0.02 of its character-based result.Source: §D.3 · pp. 12–13
  • Only two of 15 content probes survive the decode-impossible floor, despite 14 being licensed by the permutation test.The floor is set at mean +2 SD over control encodings the model demonstrably cannot decode.Source: §D.3 · p. 13
  • Corrected decode-and-restate scoring changes 328 cells from “could not decode” to “decoded” after near-verbatim answers were under-counted.An unlicensed probe is now recorded as a declared hole rather than a negative verdict.Source: §D.3 · p. 13
  • Holding items out drops the transferred probe to 0.618–0.811, below each model’s within-plaintext baseline; the authors withdraw the result because the original fit read item identity.The seen-item advantage was +0.19 to +0.38, and neither the permutation test nor the decode-impossible floor caught the leakage.Source: §D.3 · pp. 13–14
  • Per-item verdicts exist for only the encoded pipeline condition, so cross-condition contrasts use unpaired intervals despite sharing items.The authors describe this as a persistence defect; the resulting intervals are conservative rather than too narrow.Source: §D.3 · p. 14

D.4 Defects (10) and (11), and one non-defect

  • The pipeline’s echo screen had not been run, and benign-arm comprehension was unmeasured; subsequent checks reject four of nine cells and find similar decoding across arms.Across sixteen model–encoding cells, the largest benign-minus-harmful ability difference is 0.10, with fifteen within 0.02.Source: §D.4 · p. 14

E.1 The refusal judge, and run-to-run variation

  • Across 4–8 replicates per model, the harm gap moves by 0.01–0.06, while responses repeat byteidentically only 12%–58% of the time.The refusal judge is separately validated as distinguishing plaintext harmful from benign prompts on three models, with a +0.79 to +0.82 difference.Source: §E.1 · p. 14, §D.4 · p. 14
  • The refusal judge uses gpt-5-mini at temperature 0 and counts echoing or irrelevant responses as refusals.That extension is the source of the echo-scoring failure described in the instrument audit.Source: §E.1 · p. 14

E.2 Validation, held-out status, and two withdrawn inferences

Validation withdraws an apparent post-training recovery after comparison with the plaintext baseline, and reports a previously unapplied echo screen. The judge is checked against independent annotations, while the study’s thresholds and selections remain tied to the same 200 prompts.

E.2 Validation, held-out status, and two withdrawn inferences

  • The apparent encoded-benign refusal decline across stages is mostly general de-refusal: plaintext benign refusal falls in lockstep from 0.45 to 0.16, so the recovery claim is withdrawn.The encoding-specific component is flat.Source: §E.2 · pp. 14–15
  • Applying the echo screen rejects four of nine encoded cells; homoglyph clears at all three stages with displacements of 0.003–0.011.The screen requires per-item verdicts from both arms, which the original pipeline experiment did not record.Source: §E.2 · p. 15
  • Against the judge’s refusal definition, independent annotators agree with the judge at 0.90–1.00 per stratum, while 27 of 200 items are excluded by one family.The judge remains an LLM judging model responses, and echo or irrelevance is counted as refusal under the shared rubric.Source: §E.2 · p. 15
  • No result is evaluated on held-out data: all 200 prompts also determine thresholds, rungs, layers, and positions.The authors identify rung selection as the generalisation most threatened by this reuse.Source: §E.2 · p. 15

F.1 Four further limitations

The limitations leave several interpretations and mechanisms unresolved, while locating the study’s evidence in a 100-item-per-arm corpus and a specific model scale. The attempted internal probe does not establish whether harm information is degraded or remains intact but unused.

F.1 Four further limitations

  • Differences below roughly 0.15 are not separable at n=100 per cell, and the observed run-to-run spread is 0.01–0.06.The authors caution that this cancellation pattern is specific to these conditions, not guaranteed elsewhere.Source: §F.1 · p. 15
  • At encoded harmful refusal rates of 0.91–0.99, equalisation and ceiling saturation both fit; n=100 cannot distinguish them.The authors say resolution would require lower encoded harmful refusal or a graded refusal measure, neither of which they have.Source: §F.1 · p. 15
  • Whether harm information is degraded or intact but unread remains open; held-out transfer falls below each model’s plaintext baseline but does not settle the distinction.Separating the accounts requires more items and a causal arm that the study did not run.Source: §F.1 · p. 15

F.2 The internals attempt we withdrew

  • The transferred probe is correlational, and its cross-condition result was withdrawn; the paper therefore reports behaviour rather than mechanism.The authors identify as open whether a protocol disrupts downstream use of harm information or degrades its representation, which would call for different remedies.Source: §F.2 · p. 16
Loading 2609.26176v2…