Source-linked AI summary

SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure

Qi Wang, Chengcheng Wan, Jiangtao Wang

arXiv:2609.06540v1cs.CRcs.AI

TL;DR

Jailbreak attacks can hide harmful intent in contextual wrappers, while defenses may either miss disguised attacks or over-refuse legitimate requests. SRD-GUARD rewrites prompts, jointly scores the original and rewrites, and routes requests using absolute and relative risk. It achieves average DSRs of 91.44% and 100% with ORRs of 8.00% and 12.00% on the two target models.

  • Problem

    Jailbreak defenses must distinguish concealed harmful intent from legitimate sensitive or dual-use requests while balancing DSR against ORR.

    Source: §1.1 · p. 2
  • Method

    SRD-GUARD generates five semantically faithful rewrites, jointly scores all representations with four independent evaluators, and routes requests using absolute and relative risk.

    Source: §4.1 · p. 4, §8 · p. 12
  • Results

    91.44% average DSR on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash accompany ORRs of 8.00% and 12.00%, respectively.

    Source: §8 · p. 12
  • Takeaways & Limitations

    The results support semantic rewriting, joint scoring, and risk-adaptive routing as complementary components for selective black-box jailbreak defense.

    Source: §5.4 · p. 10, §8 · pp. 12–13

    SRD-GUARD cannot simultaneously achieve zero over-refusal and perfect defense success because harmfulness and benign utility form a continuous spectrum.

    Source: §6 · p. 12

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-playing, fictional scenarios, or seemingly benign motivations. Existing inference-time defenses may miss disguised attacks or excessively refuse legitimate requests. We propose SRD-GUARD, a parameter-free, black-box defense framework that exposes concealed intent through semantic rewriting and consensus-based risk assessment. Given an input prompt, SRD-GUARD generates five semantically related rewrites that preserve the underlying objective while removing unnecessary contextual packaging. The original prompt and rewrites are jointly evaluated by multiple independent LLM-based safety scorers on a continuous risk scale. A decision module combines absolute risk thresholds with relative risk changes between the original and rewritten prompts to adaptively intercept, preserve, or warn on requests. We evaluate SRD-GUARD against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored and DeepSeek-V4-Flash using AdvBench and OR-Bench-Hard. SRD-GUARD achieves average DSRs of 91.44% and 100%, with ORRs of 8.00% and 12.00%, respectively. Compared with evaluated baselines, it provides a more favorable DSR--ORR trade-off. Ablation studies show that rewriting exposes concealed harmful intent, joint scoring improves robustness to individual evaluator behavior, and risk-adaptive decision making enables selective handling of ambiguous inputs. These results demonstrate that semantic intent exposure, consensus-based risk assessment, and relative-risk-aware routing provide an effective and selective approach to black-box jailbreak defense. The artifact is available at https://anonymous.4open.science/status/CICD-Guard-D648.

1 INTRODUCTION

Jailbreak prompts can conceal harmful intent in contextual packaging, creating a tension between blocking attacks and preserving legitimate requests. SRD-GUARD addresses this by rewriting prompts, jointly scoring multiple representations, and routing interventions adaptively.

  • Contextual wrappers such as role-playing, fictional scenarios, and defensive or research motivations can obscure harmful prompt intent.Source: §1.1 · p. 2
  • Effective jailbreak defense must balance protection against malicious requests with unnecessary refusal of benign or dual-use requests.Source: §1.1 · p. 2
  • SRD-GUARD generates semantically faithful rewrites that reduce unnecessary contextual wrapping while preserving information essential to the request’s meaning.Source: §1.2 · p. 3
  • The framework jointly scores original and rewritten prompts with independent evaluators and uses relative risk to adapt routing rather than applying uniform rejection.Source: §1.2 · p. 3
  • SRD-GUARD improves defense effectiveness with limited over-refusal increases across the evaluated target models and attacks.On Llama-3-8B-Uncensored, ORR rises by 2.33 percentage points while DSR improves by 70.06–89.07 percentage points; on DeepSeek-V4-Flash, the corresponding changes are 2.00 and 14.83–52.50 percentage points.Source: §1.2 · p. 3

2 BACKGROUND

Prompt-based attacks manipulate natural-language context to induce unsafe behavior, often embedding harmful requests in role-playing or fictional settings. Existing defenses operate at input, model, or output stages, with different retraining and computational requirements.

  • Prompt-based attacks exploit instruction-following behavior through crafted natural-language inputs rather than model parameters or internal representations.Source: §2 · p. 3
  • Attackers commonly hide harmful requests inside role-playing, fictional, hypothetical, or other contextual packaging.Source: §2 · p. 3
  • Input-level defenses inspect or transform prompts before target-model processing, whereas model-level defenses modify parameters or safety behavior through training.Source: §2 · pp. 3–4
  • Output-level defenses examine generated responses and suppress or modify potentially unsafe content.Source: §2 · p. 4

3 THREAT MODEL

The threat model considers black-box attackers who use public interfaces and contextual manipulation to elicit harmful content. SRD-GUARD is an external, parameter-free layer that operates without access to target-model internals.

  • The adversary accesses the target LLM only through its public interaction interface.Source: §3.1 · p. 4
  • The adversary may use role-playing, fictional scenarios, hypothetical settings, indirect instructions, and other packaging to obscure harmful intent.Source: §3.1 · p. 4
  • SRD-GUARD sits between the user and target model without modifying the target model’s parameters, architecture, or inference procedure.Source: §3.2 · p. 4
  • The defense uses only the input prompt and auxiliary rewriting and scoring outputs, enabling application across target LLMs without retraining them.Source: §3.2 · p. 4

4 METHOD

SRD-GUARD exposes concealed intent through five semantic rewrites, evaluates original and rewritten prompts with multi-model risk scoring, and applies threshold- and relative-risk-based routing. The decision process intercepts high-risk requests, passes consistently low-risk requests, and warns or rewrites gray-zone cases.

4 METHOD

  • SRD-GUARD generates five semantically related variants that preserve the original objective while removing unnecessary contextual packaging.Source: §4.1 · p. 4, §4.2 · p. 5
  • Multiple rewritten representations reduce dependence on a single paraphrase and test whether elevated risk persists across formulations.Source: §4.2 · p. 5
  • Four independent safety scorers assign continuous 0–10 risk scores to the original prompt and each rewrite, whose assessments are aggregated for consensus.Source: §4.3 · pp. 5–6
  • The scoring module produces original, individual-variant, and overall rewritten-variant risks for downstream decision making.Source: §4.3 · p. 6
  • Requests are intercepted when the original or any rewritten variant reaches the danger threshold S_i ≥ 6.0.Source: §4.4 · p. 6
  • Requests below 3.0 for both the original and every rewrite pass unchanged, while gray-zone cases use relative risk to select a warning or rewritten response.Source: §4.4 · p. 6

5 EVALUATION

The evaluation tests SRD-GUARD’s defense effectiveness, over-refusal, component contributions, and sensitivity across harmful and benign-request benchmarks, attack methods, and target models with different intrinsic safety behavior.

  • The evaluation addresses defense effectiveness, over-refusal, ablation contributions, and sensitivity to scoring-model and rewrite-variant counts.Source: §5 · p. 7
  • AdvBench measures harmful requests, while transformed AdvBench prompts assess robustness against black-box jailbreak manipulation.Source: §5.1.1 · p. 7
  • OR-Bench-Hard evaluates whether the defense distinguishes genuinely harmful intent from benign requests involving sensitive topics.Source: §5.1.1 · p. 7
  • The study evaluates 300 AdvBench prompts and 300 OR-Bench-Hard prompts under the same defense and target-model configuration.Source: §5.1.1 · p. 7
  • Llama-3-8B-Uncensored and DeepSeek-V4-Flash provide target models with substantially different intrinsic safety behaviors.Source: §5.1.2 · p. 7
  • SRD-GUARD uses parameter-free semantic rewriting and four independent lightweight open-source LLM scoring models, while comparisons include DPP, IA, and SR.Source: §5.1.3 · p. 8, §5.1.4 · p. 8

5.2 ANSWER TO RQ1 (DEFENSE EFFECTIVENESS)

SRD-GUARD maintains high defense success across UNIATTACK, CIPHER, and DeepInception, including 100% DSR across all three attacks on DeepSeek-V4-Flash.

  • SRD-GUARD achieves 91.00%, 85.00%, and 98.33% DSR against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored.Source: §5.2 · p. 8
  • On DeepSeek-V4-Flash, SRD-GUARD achieves 100% DSR across UNIATTACK, CIPHER, and DeepInception.Source: §5.2 · p. 8
  • SRD-GUARD outperforms DPP and SR on Llama-3-8B-Uncensored while remaining competitive with IA across the three attacks.Source: §5.2 · p. 8
  • SRD-GUARD improves DSR over the strongest baseline by 16.67 percentage points on CIPHER and 10.77 percentage points on DeepInception.Source: §5.2 · p. 8
  • The defense combines semantic rewriting, joint scoring, and absolute-plus-relative risk routing to detect concealed harmful requests.Source: §5.2 · p. 9
  • SRD-GUARD increases ORR by 2.33% on Llama-3-8B-Uncensored and 2.00% on DeepSeek-V4-Flash while improving DSR across the attacks.Source: §5.2 · p. 9

5.3 ANSWER TO RQ2 (OVER-REFUSAL)

SRD-GUARD achieves a favorable defense–utility balance, combining low over-refusal with strong DSR; ablations show that rewriting, joint scoring, and risk-adaptive decisions contribute complementarily.

  • SRD-GUARD records 8.00% ORR on Llama-3-8B-Uncensored and 12.00% on DeepSeek-V4-Flash, below IA’s 45.67% and 38.33%.Source: §5.3 · p. 9
  • On Llama-3-8B-Uncensored, SRD-GUARD pairs DSRs of 91.00%, 85.00%, and 98.33% with 8.00% ORR across the three attacks.Source: §5.3 · p. 9
  • SRD-GUARD improves average DSR by 30.30% on Llama-3-8B-Uncensored and 15.67% on DeepSeek-V4-Flash while reducing ORR relative to baselines.Source: §5.3 · p. 10

Ablation study

  • The complete system achieves the highest average DSR and lowest ORR across both target models, while removing any component degrades one or both objectives.Source: §5.4 · p. 10
  • Removing rewriting drops average DSR from 91.44% to 50.66% on Llama-3-8B-Uncensored and from 100% to 64.52% on DeepSeek-V4-Flash.Source: §5.4 · p. 10
  • Replacing joint scoring with one Llama3-8B scorer raises ORR to 37% and 50% on the two target models despite an apparent DSR increase in one setting.Source: §5.4 · p. 10
  • Removing the Decision Module lowers average DSR to 79.33% and 79.50% while raising ORR to 24.50% and 32.50%.Source: §5.4 · p. 11
  • The three components have complementary roles: rewriting exposes hidden intent, joint scoring stabilizes risk estimation, and decision routing controls over-refusal.Source: §5.4 · p. 11

5.5 ANSWER TO RQ4 (SENSITIVITY)

Sensitivity analysis identifies five rewritten variants and four scoring models as the strongest configuration, balancing defense success against over-refusal.

  • 85% DSR and 8% ORR make five rewritten variants the strongest tested configuration.Seven variants reduce DSR to 82% and increase ORR to 12%.Source: §5.5 · p. 11
  • 85% DSR and 8% ORR make four scorers the strongest tested scoring configuration.The passage states that using fewer scorers produces weaker performance.Source: §5.5 · p. 11
  • Table 2 reports sensitivity analysis on Llama-3-8B-Uncensored under CIPHER.Source: Table 2 · p. 11, §5.5 · p. 11

6 DISCUSSION

The discussion identifies deployment cost and unresolved safety-boundary ambiguity as important constraints on SRD-GUARD’s practical use.

Deployment Overhead

  • Four independent LLM evaluators create substantial memory and computation overhead during deployment.They score both the original prompt and rewritten variants using continuous risk scores.Source: §6 · p. 11

Ambiguity of Safety Boundaries

  • OR-Bench’s sensitive, contextually legitimate prompts make a sharp harmfulness boundary difficult to establish.Source: §6 · pp. 11–12
  • SRD-GUARD cannot simultaneously achieve zero over-refusal and perfect defense success on borderline requests.Increasing sensitivity to concealed harmful intent may affect some benign requests, while relaxing the defense may miss some harmful requests.Source: §6 · p. 12

7 RELATED WORK

Related defenses use perturbation, intention analysis, or multi-model judging, but SRD-GUARD combines multiple semantic representations with continuous, relative-risk-aware scoring.

  • Input-level defenses such as SmoothLLM primarily manipulate suspicious prompts and may increase over-refusal through conservative treatment.Source: §7 · p. 12
  • Intention Analysis infers a single essential intention rather than comparing the original prompt with multiple semantic representations.Source: §7 · p. 12
  • Existing multi-model defenses generally assess one prompt representation with absolute judgments, whereas SRD-GUARD jointly scores originals and rewrites.SRD-GUARD adds continuous risk scoring and explicit relative-risk comparison.Source: §7 · p. 12

8 CONCLUSION

SRD-GUARD exposes concealed intent through semantic rewriting, aggregates independent safety judgments, and routes requests using absolute and relative risk. Across target models, it achieves strong defense effectiveness while preserving benign-request utility more selectively than uniformly restrictive defenses.

  • SRD-GUARD is a parameter-free, black-box framework that uses semantic rewriting, joint scoring, and risk-adaptive routing.The decision module combines absolute risk with relative risk changes between original and rewritten prompts.Source: §8 · p. 12
  • 91.44% average DSR on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash accompany ORRs of 8.00% and 12.00%, respectively.Source: §8 · p. 12
  • The framework targets a more favorable balance between defense effectiveness and benign-request preservation without modifying the target model.Source: §8 · p. 13
Loading 2609.06540v1…