PaperlayerView exact revision on arXiv

Source-linked AI summary

SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure

Qi Wang, Chengcheng Wan, Jiangtao Wang

arXiv:2609.06540v1cs.CRcs.AI

TL;DR

Jailbreak attacks can hide harmful intent in contextual wrappers, while defenses may either miss disguised attacks or over-refuse legitimate requests. SRD-GUARD rewrites prompts, jointly scores the original and rewrites, and routes requests using absolute and relative risk. It achieves average DSRs of 91.44% and 100% with ORRs of 8.00% and 12.00% on the two target models.

  • Problem

    Jailbreak defenses must distinguish concealed harmful intent from legitimate sensitive or dual-use requests while balancing DSR against ORR.

  • Method

    SRD-GUARD generates five semantically faithful rewrites, jointly scores all representations with four independent evaluators, and routes requests using absolute and relative risk.

  • Results

    91.44% average DSR on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash accompany ORRs of 8.00% and 12.00%, respectively.

  • Takeaways & Limitations

    The results support semantic rewriting, joint scoring, and risk-adaptive routing as complementary components for selective black-box jailbreak defense.

  • Takeaways & Limitations

    SRD-GUARD cannot simultaneously achieve zero over-refusal and perfect defense success because harmfulness and benign utility form a continuous spectrum.

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-playing, fictional scenarios, or seemingly benign motivations. Existing inference-time defenses may miss disguised attacks or excessively refuse legitimate requests. We propose SRD-GUARD, a parameter-free, black-box defense framework that exposes concealed intent through semantic rewriting and consensus-based risk assessment. Given an input prompt, SRD-GUARD generates five semantically related rewrites that preserve the underlying objective while removing unnecessary contextual packaging. The original prompt and rewrites are jointly evaluated by multiple independent LLM-based safety scorers on a continuous risk scale. A decision module combines absolute risk thresholds with relative risk changes between the original and rewritten prompts to adaptively intercept, preserve, or warn on requests. We evaluate SRD-GUARD against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored and DeepSeek-V4-Flash using AdvBench and OR-Bench-Hard. SRD-GUARD achieves average DSRs of 91.44% and 100%, with ORRs of 8.00% and 12.00%, respectively. Compared with evaluated baselines, it provides a more favorable DSR--ORR trade-off. Ablation studies show that rewriting exposes concealed harmful intent, joint scoring improves robustness to individual evaluator behavior, and risk-adaptive decision making enables selective handling of ambiguous inputs. These results demonstrate that semantic intent exposure, consensus-based risk assessment, and relative-risk-aware routing provide an effective and selective approach to black-box jailbreak defense. The artifact is available at https://anonymous.4open.science/status/CICD-Guard-D648.

1 INTRODUCTION

Jailbreak prompts can conceal harmful intent in contextual packaging, creating a tension between blocking attacks and preserving legitimate requests. SRD-GUARD addresses this by rewriting prompts, jointly scoring multiple representations, and routing interventions adaptively.

  • Contextual wrappers such as role-playing, fictional scenarios, and defensive or research motivations can obscure harmful prompt intent.
  • Effective jailbreak defense must balance protection against malicious requests with unnecessary refusal of benign or dual-use requests.
  • SRD-GUARD generates semantically faithful rewrites that reduce unnecessary contextual wrapping while preserving information essential to the request’s meaning.
  • The framework jointly scores original and rewritten prompts with independent evaluators and uses relative risk to adapt routing rather than applying uniform rejection.
  • SRD-GUARD improves defense effectiveness with limited over-refusal increases across the evaluated target models and attacks.On Llama-3-8B-Uncensored, ORR rises by 2.33 percentage points while DSR improves by 70.06–89.07 percentage points; on DeepSeek-V4-Flash, the corresponding changes are 2.00 and 14.83–52.50 percentage points.

2 BACKGROUND

Prompt-based attacks manipulate natural-language context to induce unsafe behavior, often embedding harmful requests in role-playing or fictional settings. Existing defenses operate at input, model, or output stages, with different retraining and computational requirements.

  • Prompt-based attacks exploit instruction-following behavior through crafted natural-language inputs rather than model parameters or internal representations.
  • Attackers commonly hide harmful requests inside role-playing, fictional, hypothetical, or other contextual packaging.
  • Input-level defenses inspect or transform prompts before target-model processing, whereas model-level defenses modify parameters or safety behavior through training.
  • Output-level defenses examine generated responses and suppress or modify potentially unsafe content.

3 THREAT MODEL

The threat model considers black-box attackers who use public interfaces and contextual manipulation to elicit harmful content. SRD-GUARD is an external, parameter-free layer that operates without access to target-model internals.

  • The adversary accesses the target LLM only through its public interaction interface.
  • The adversary may use role-playing, fictional scenarios, hypothetical settings, indirect instructions, and other packaging to obscure harmful intent.
  • SRD-GUARD sits between the user and target model without modifying the target model’s parameters, architecture, or inference procedure.
  • The defense uses only the input prompt and auxiliary rewriting and scoring outputs, enabling application across target LLMs without retraining them.

4 METHOD

SRD-GUARD exposes concealed intent through five semantic rewrites, evaluates original and rewritten prompts with multi-model risk scoring, and applies threshold- and relative-risk-based routing. The decision process intercepts high-risk requests, passes consistently low-risk requests, and warns or rewrites gray-zone cases.

  • 4 METHOD: SRD-GUARD generates five semantically related variants that preserve the original objective while removing unnecessary contextual packaging.
  • 4 METHOD: Multiple rewritten representations reduce dependence on a single paraphrase and test whether elevated risk persists across formulations.
  • 4 METHOD: Four independent safety scorers assign continuous 0–10 risk scores to the original prompt and each rewrite, whose assessments are aggregated for consensus.
  • 4 METHOD: The scoring module produces original, individual-variant, and overall rewritten-variant risks for downstream decision making.
  • 4 METHOD: Requests are intercepted when the original or any rewritten variant reaches the danger threshold S_i ≥ 6.0.
  • 4 METHOD: Requests below 3.0 for both the original and every rewrite pass unchanged, while gray-zone cases use relative risk to select a warning or rewritten response.

5 EVALUATION

The evaluation tests SRD-GUARD’s defense effectiveness, over-refusal, component contributions, and sensitivity across harmful and benign-request benchmarks, attack methods, and target models with different intrinsic safety behavior.

  • The evaluation addresses defense effectiveness, over-refusal, ablation contributions, and sensitivity to scoring-model and rewrite-variant counts.
  • AdvBench measures harmful requests, while transformed AdvBench prompts assess robustness against black-box jailbreak manipulation.
  • OR-Bench-Hard evaluates whether the defense distinguishes genuinely harmful intent from benign requests involving sensitive topics.
  • The study evaluates 300 AdvBench prompts and 300 OR-Bench-Hard prompts under the same defense and target-model configuration.
  • Llama-3-8B-Uncensored and DeepSeek-V4-Flash provide target models with substantially different intrinsic safety behaviors.
  • SRD-GUARD uses parameter-free semantic rewriting and four independent lightweight open-source LLM scoring models, while comparisons include DPP, IA, and SR.

5.2 ANSWER TO RQ1 (DEFENSE EFFECTIVENESS)

SRD-GUARD maintains high defense success across UNIATTACK, CIPHER, and DeepInception, including 100% DSR across all three attacks on DeepSeek-V4-Flash.

  • SRD-GUARD achieves 91.00%, 85.00%, and 98.33% DSR against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored.
  • On DeepSeek-V4-Flash, SRD-GUARD achieves 100% DSR across UNIATTACK, CIPHER, and DeepInception.
  • SRD-GUARD outperforms DPP and SR on Llama-3-8B-Uncensored while remaining competitive with IA across the three attacks.
  • SRD-GUARD improves DSR over the strongest baseline by 16.67 percentage points on CIPHER and 10.77 percentage points on DeepInception.
  • The defense combines semantic rewriting, joint scoring, and absolute-plus-relative risk routing to detect concealed harmful requests.
  • SRD-GUARD increases ORR by 2.33% on Llama-3-8B-Uncensored and 2.00% on DeepSeek-V4-Flash while improving DSR across the attacks.

5.3 ANSWER TO RQ2 (OVER-REFUSAL)

SRD-GUARD achieves a favorable defense–utility balance, combining low over-refusal with strong DSR; ablations show that rewriting, joint scoring, and risk-adaptive decisions contribute complementarily.

  • SRD-GUARD records 8.00% ORR on Llama-3-8B-Uncensored and 12.00% on DeepSeek-V4-Flash, below IA’s 45.67% and 38.33%.
  • On Llama-3-8B-Uncensored, SRD-GUARD pairs DSRs of 91.00%, 85.00%, and 98.33% with 8.00% ORR across the three attacks.
  • SRD-GUARD improves average DSR by 30.30% on Llama-3-8B-Uncensored and 15.67% on DeepSeek-V4-Flash while reducing ORR relative to baselines.
  • Ablation study: The complete system achieves the highest average DSR and lowest ORR across both target models, while removing any component degrades one or both objectives.
  • Ablation study: Removing rewriting drops average DSR from 91.44% to 50.66% on Llama-3-8B-Uncensored and from 100% to 64.52% on DeepSeek-V4-Flash.
  • Ablation study: Replacing joint scoring with one Llama3-8B scorer raises ORR to 37% and 50% on the two target models despite an apparent DSR increase in one setting.
  • Ablation study: Removing the Decision Module lowers average DSR to 79.33% and 79.50% while raising ORR to 24.50% and 32.50%.
  • Ablation study: The three components have complementary roles: rewriting exposes hidden intent, joint scoring stabilizes risk estimation, and decision routing controls over-refusal.

5.5 ANSWER TO RQ4 (SENSITIVITY)

Sensitivity analysis identifies five rewritten variants and four scoring models as the strongest configuration, balancing defense success against over-refusal.

  • 85% DSR and 8% ORR make five rewritten variants the strongest tested configuration.Seven variants reduce DSR to 82% and increase ORR to 12%.
  • 85% DSR and 8% ORR make four scorers the strongest tested scoring configuration.The passage states that using fewer scorers produces weaker performance.
  • Table 2 reports sensitivity analysis on Llama-3-8B-Uncensored under CIPHER.

6 DISCUSSION

The discussion identifies deployment cost and unresolved safety-boundary ambiguity as important constraints on SRD-GUARD’s practical use.

  • Deployment Overhead: Four independent LLM evaluators create substantial memory and computation overhead during deployment.They score both the original prompt and rewritten variants using continuous risk scores.
  • Ambiguity of Safety Boundaries: OR-Bench’s sensitive, contextually legitimate prompts make a sharp harmfulness boundary difficult to establish.
  • Ambiguity of Safety Boundaries: SRD-GUARD cannot simultaneously achieve zero over-refusal and perfect defense success on borderline requests.Increasing sensitivity to concealed harmful intent may affect some benign requests, while relaxing the defense may miss some harmful requests.

7 RELATED WORK

Related defenses use perturbation, intention analysis, or multi-model judging, but SRD-GUARD combines multiple semantic representations with continuous, relative-risk-aware scoring.

  • Input-level defenses such as SmoothLLM primarily manipulate suspicious prompts and may increase over-refusal through conservative treatment.
  • Intention Analysis infers a single essential intention rather than comparing the original prompt with multiple semantic representations.
  • Existing multi-model defenses generally assess one prompt representation with absolute judgments, whereas SRD-GUARD jointly scores originals and rewrites.SRD-GUARD adds continuous risk scoring and explicit relative-risk comparison.

8 CONCLUSION

SRD-GUARD exposes concealed intent through semantic rewriting, aggregates independent safety judgments, and routes requests using absolute and relative risk. Across target models, it achieves strong defense effectiveness while preserving benign-request utility more selectively than uniformly restrictive defenses.

  • SRD-GUARD is a parameter-free, black-box framework that uses semantic rewriting, joint scoring, and risk-adaptive routing.The decision module combines absolute risk with relative risk changes between original and rewritten prompts.
  • 91.44% average DSR on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash accompany ORRs of 8.00% and 12.00%, respectively.
  • The framework targets a more favorable balance between defense effectiveness and benign-request preservation without modifying the target model.
Loading 2609.06540v1…