Source-linked AI summary
Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts
Yongxi Zhou, Wenbo Ye, Yuanzhe Liu, Zihan Dong, Junwei Yao
TL;DR
Safety evaluation depends on judges whose verdicts may respond to style rather than fixed harmful content. The paper tests this with content-preserving wrappers and paired, noise-aware scoring, finding narrow but exploitable judge-specific blind spots and unstable leaderboard rankings.
Problem
The paper asks whether automatic safety judges grade a reply’s content consistently when only its surrounding style changes.
Method
The study applies fixed wrappers to 600 JailbreakBench responses, evaluates eight judges with paired tests and measured noise floors, and validates flips with human annotation.
Results
Specific judges show substantial wrapper sensitivity, including 19.9% GPT-4o-mini flips, 12.3% Llama Guard 4 flips, and a tenfold reduction under a StrongREJECT-style prompt.
Takeaways & Limitations
Safety scores should report a flip budget because judge-specific stylistic bias can affect ASR, defense, and leaderboard measurements.
Takeaways & Limitations
The nine wrappers characterize this response set rather than providing a population estimate, and leaderboard reversals remain unconfirmed on its compressed baselines.
Abstract
from arXiv · showhide
Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers behind almost every reported jailbreak success rate, defense evaluation, and safety leaderboard. We ask whether these judges grade what a reply contains or how it sounds. We keep a reply's content fixed and add content-invariant style wrappers: fixed strings placed before or after the reply that change only its tone (an educational disclaimer, a fake safety "reasoning" block, a token refusal followed by the unchanged harmful body), or, on harmless refusals, framing that merely sounds dangerous. The body is preserved byte-for-byte, so a faithful judge must return the same verdict, and any flip is an error of the judge, not a change in safety. Over 600 JailbreakBench replies x up to 7 forms x 8 judges, we measure flip rates with paired significance tests and measured noise floors. Findings are precise rather than universal: most judges barely move, but specific judges harbor cheaply exploitable blind spots. A token-refusal wrapper flips 19.9% of GPT-4o-mini's correct "unsafe" verdicts (noise floor 0.5%; 18.2% under majority-of-three re-scoring) yet moves Claude only 0.4%. The deployed Llama Guard 4 is deterministically gamed: an "educational course" framing flips 12.3% of its harmful verdicts to safe. A second deployed guard (gpt-oss-safeguard-20b) is immune, and rewriting only the grading prompt (StrongREJECT-style) cuts the attack tenfold on the identical model -- the vulnerability lives in the judge, not the content. A two-annotator human validation confirms 100% content invariance and 90% of flips as judge errors (kappa 0.95-1.0), and a bootstrap shows the underlying model ranking is already unstable to sampling alone. We release the dataset, wrappers, code, and per-verdict labels.
1 Introduction
The paper tests whether safety judges preserve verdicts when response content is fixed but style changes. It reports judge-specific blind spots rather than claiming that safety judges generally prioritize style over substance.
- The study asks whether changing only a response’s style can change a safety verdict, which a faithful judge should prevent.
- The attack targets judges rather than models and preserves response content by construction, avoiding intent-preservation confounds from rewriting.
- The evaluation uses paired flip rates, significance tests, bootstrap intervals, and measured noise floors across multiple judges and wrappers.
- The paper finds that ranking reorderings on the jailbreak-artifact set cannot be confirmed because model rankings are unstable under judge choice and sampling alone.
- Human-anchored validation treats content-preserving wrapper flips as judge errors and accompanies the study with a public release.
- A wrapper is a fixed prefix or suffix that changes a reply’s tone without changing its content; flips compare verdicts on the original and wrapped forms.
2 Related Work
Prior work establishes that LLM judges can be biased, inconsistent, and sensitive to surface form, while safety evaluation itself suffers from validity concerns. This paper shifts those concerns to the scoring side by holding response content fixed and varying only surrounding style.
- General LLM evaluation studies document position, verbosity, self-enhancement, fluency, and reasoning-style biases in automatic judging.
- Unlike token-injection attacks, the paper uses fixed human-readable wrappers to test safety-verdict sensitivity without modifying the response body.
- The study operationalizes run-to-run inconsistency as a noise floor and treats judge flip rate as one structured component of label noise.
- Earlier safety work shows that request reformulation and operational framing can alter model behavior; this paper applies the analogous concern to scoring while holding content fixed.
- Concerns about invalid attack-success comparisons, evaluator variance, and spurious evaluation signals motivate examining judge reliability directly.
- Guard-model research typically hardens guards against inputs, whereas this paper attacks their scoring behavior and also probes false positives on refusals.
3 Method
The method evaluates fixed response triples with content-invariant wrappers across a heterogeneous judge panel. Paired scoring, corrected tests, bootstrap intervals, and repeated runs distinguish wrapper effects from ordinary judge noise.
- Each data unit is a fixed behavior–response–gold triple, and the response is never edited during evaluation.
- Wrappers prepend or append fixed strings while preserving the response body verbatim; harmful replies receive harm-hiding wrappers and refusals receive harm-inventing wrappers.
- The panel contains a keyword baseline, GPT-4o-mini under two prompts, three chat judges, Llama Guard 4, and gpt-oss-safeguard-20b.
- Flip rates condition on correct original-form judgments and measure harmful-to-safe and refusal-to-unsafe changes separately.
- Exact McNemar tests with Holm and Benjamini–Hochberg correction, 5,000-resample bootstrap intervals, and repeated scoring define the significance and noise procedures.
4 Experimental Setup
The experiments use 600 balanced JailbreakBench responses spanning attack methods, target models, and harm categories. Eight API-served judges score them under a repeated, corrected evaluation protocol with human validation.
- The response set contains 600 items split evenly between unsafe compliances and refusals from public JailbreakBench artifacts.
- The set is balanced across four target models and ten harm categories, allowing it to function as a mini leaderboard.
- Eight judges include a keyword baseline, two GPT-4o-mini prompts, three chat models, Llama Guard 4, and gpt-oss-safeguard-20b.
- The protocol uses temperature 0, three original-form repeats, exact McNemar tests with Holm correction, and 5,000-sample bootstrap confidence intervals.
- A 145-item two-annotator human validation supplements the automated evaluation, while the reported API cost was under $5.
5 Results
Across eight judges and 600 JailbreakBench responses, content-invariant wrappers produced judge-specific flips, while human validation confirmed that most observed flips were judge errors. The same experiments also found that ranking claims on this artifact set are unstable to sampling and judge choice.
- 19.9% of GPT-4o-mini’s correct unsafe verdicts flipped under refusal_then_comply, versus 0.4% for Claude.The GPT-4o-mini result exceeded its 0.5% noise floor.
- 12.3% of Llama Guard 4’s harmful verdicts flipped under educational_framing, while gpt-oss-safeguard-20b remained within its noise floor.The deployed guards therefore showed sharply different susceptibility to the same class of restyling.
- Tone-only wrappers flipped 0.0–2.1% of correct verdicts per judge, including 2.1% for GPT-4o-mini and 1.7% for Llama Guard 4.These effects remained above measured noise floors for the two vulnerable judges.
- Assertion-adding wrappers flipped 2–6× more than tone-only wrappers on every judge with a measurable effect.The largest headline effects therefore occurred when wrappers introduced contestable statements, although tone-only wrappers still affected vulnerable judges.
- 18.2% of GPT-4o-mini’s refusal_then_comply flips persisted under majority-of-three rescoring, while 11.9% of Llama Guard 4’s educational_framing cases flipped on all three repeats.Rescoring reduced neither attack to its original-form noise floor, and Llama Guard 4 was nearly deterministic on wrapped inputs.
- A StrongREJECT-style grading prompt reduced GPT-4o-mini’s refusal_then_comply flip rate from 19.9% to 1.7% on the same model.This isolates the vulnerability to judge design rather than the judged response content.
- The baseline safest-model ranking was stable in only 39–70% of bootstrap resamples for six of seven LLM judges.Wrapper-induced reorderings could not be distinguished from sampling noise on the compressed artifact set.
- Human validation found 100% content invariance and confirmed 90% of flips as judge errors.The validation used 145 items, with κ = 1.0 for content invariance and κ = 0.949 for harm labels.
6 Discussion
Specific safety judges can be shifted by one-sentence wrappers at substantial rates, introducing stylistic bias into reported evaluation scores. The paper therefore recommends measuring score stability under content-preserving restyling rather than discarding judges altogether.
- 6 Discussion: 8–20% wrapper-induced verdict changes on particular judges can introduce stylistic bias into ASR, defense, and leaderboard measurements.The bias is judge-specific and may not cancel across papers using different judges.
- 6 Discussion: A reported flip budget should accompany safety scores to show how much judgments move when responses are restyled without changing content.The paper compares this to reporting instrument error in physical measurement.
- 6 Discussion: Content-focused rubric judging and majority-vote panels are proposed as natural mitigations for judge gameability.Cross-judge results indicate that the strongest content-only rubric is least gameable, while one deployed guard is immune.
7 Limitations
The study characterizes nine wrappers rather than estimating a population-wide effect, and several design and data constraints limit broader conclusions. In particular, leaderboard reversals remain unconfirmed, some wrapper effects are only suggestive, and the evaluation covers single-turn text responses.
- 7 Limitations: Nine wrappers characterize this set rather than providing a population estimate, with the largest effect coming from an overtly adversarial wrapper.Subtle harm-hiding effects exceed the noise floor in confidence intervals but do not survive Holm correction at n ≈291.
- 7 Limitations: Leaderboard reorderings remain unconfirmed because baseline unsafe rates are compressed, so affirmative reversal claims require responses with wider separation.The bootstrap analysis treats observed reorderings as unconfirmed.
- 7 Limitations: The style label is contestable for assertion-adding wrappers, making the tone-only stratum the stronger evidence.A token refusal or false course context changes what the message asserts, and one wrapper produced the sole annotator disagreement.
- 7 Limitations: The study covers single-turn text responses, while multi-turn and agent-trajectory judging remain future work.This limits direct extension of the findings to interactive or agentic evaluation settings.
8 Ethics
The paper targets the evaluation layer to make safety measurement more honest rather than to jailbreak models. Because the wrappers can evade a deployed moderation filter, the authors report specific results to affected developers and release them to support defense auditing.
- 8 Ethics: The study attacks the evaluation layer to improve safety measurement, not to jailbreak models.The responses come from existing public benchmarks, and the shipped demo uses non-operational placeholder text.
- 8 Ethics: Generic human-readable wrappers are trivially rediscoverable, so withholding them would impede defenders auditing their own pipelines more than attackers.The authors frame disclosure as supporting hardening of scoring layers.
- 8 Ethics: The authors report Llama Guard 4 evasion results to Meta and GPT-4o-mini destabilization results to OpenAI before wide release.The wrappers confer no new harmful capability to a model but do constitute working evasion framings against a deployed filter.
A Extended related work
The released repository includes a full multi-venue map of related work spanning major machine-learning and language venues from 2025–26.
- A Extended related work: The released repository contains a related-work map covering COLM, ICLR, ICML, NeurIPS, ACL, EMNLP, and AAAI from 2025–26.