Source-linked AI summary
SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure
Qi Wang, Chengcheng Wan, Jiangtao Wang
TL;DR
Jailbreak attacks can hide harmful intent in contextual wrappers, while defenses may either miss disguised attacks or over-refuse legitimate requests. SRD-GUARD rewrites prompts, jointly scores the original and rewrites, and routes requests using absolute and relative risk. It achieves average DSRs of 91.44% and 100% with ORRs of 8.00% and 12.00% on the two target models.
Problem
Jailbreak defenses must distinguish concealed harmful intent from legitimate sensitive or dual-use requests while balancing DSR against ORR.
Source: §1.1 · p. 2Method
SRD-GUARD generates five semantically faithful rewrites, jointly scores all representations with four independent evaluators, and routes requests using absolute and relative risk.
Source: §4.1 · p. 4, §8 · p. 12Results
91.44% average DSR on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash accompany ORRs of 8.00% and 12.00%, respectively.
Source: §8 · p. 12Takeaways & Limitations
The results support semantic rewriting, joint scoring, and risk-adaptive routing as complementary components for selective black-box jailbreak defense.
Source: §5.4 · p. 10, §8 · pp. 12–13SRD-GUARD cannot simultaneously achieve zero over-refusal and perfect defense success because harmfulness and benign utility form a continuous spectrum.
Source: §6 · p. 12
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-playing, fictional scenarios, or seemingly benign motivations. Existing inference-time defenses may miss disguised attacks or excessively refuse legitimate requests. We propose SRD-GUARD, a parameter-free, black-box defense framework that exposes concealed intent through semantic rewriting and consensus-based risk assessment. Given an input prompt, SRD-GUARD generates five semantically related rewrites that preserve the underlying objective while removing unnecessary contextual packaging. The original prompt and rewrites are jointly evaluated by multiple independent LLM-based safety scorers on a continuous risk scale. A decision module combines absolute risk thresholds with relative risk changes between the original and rewritten prompts to adaptively intercept, preserve, or warn on requests. We evaluate SRD-GUARD against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored and DeepSeek-V4-Flash using AdvBench and OR-Bench-Hard. SRD-GUARD achieves average DSRs of 91.44% and 100%, with ORRs of 8.00% and 12.00%, respectively. Compared with evaluated baselines, it provides a more favorable DSR--ORR trade-off. Ablation studies show that rewriting exposes concealed harmful intent, joint scoring improves robustness to individual evaluator behavior, and risk-adaptive decision making enables selective handling of ambiguous inputs. These results demonstrate that semantic intent exposure, consensus-based risk assessment, and relative-risk-aware routing provide an effective and selective approach to black-box jailbreak defense. The artifact is available at https://anonymous.4open.science/status/CICD-Guard-D648.
1 INTRODUCTION
Jailbreak prompts can conceal harmful intent in contextual packaging, creating a tension between blocking attacks and preserving legitimate requests. SRD-GUARD addresses this by rewriting prompts, jointly scoring multiple representations, and routing interventions adaptively.
- Contextual wrappers such as role-playing, fictional scenarios, and defensive or research motivations can obscure harmful prompt intent.Source: §1.1 · p. 2
- Effective jailbreak defense must balance protection against malicious requests with unnecessary refusal of benign or dual-use requests.Source: §1.1 · p. 2
- SRD-GUARD generates semantically faithful rewrites that reduce unnecessary contextual wrapping while preserving information essential to the request’s meaning.Source: §1.2 · p. 3
- The framework jointly scores original and rewritten prompts with independent evaluators and uses relative risk to adapt routing rather than applying uniform rejection.Source: §1.2 · p. 3
- SRD-GUARD improves defense effectiveness with limited over-refusal increases across the evaluated target models and attacks.On Llama-3-8B-Uncensored, ORR rises by 2.33 percentage points while DSR improves by 70.06–89.07 percentage points; on DeepSeek-V4-Flash, the corresponding changes are 2.00 and 14.83–52.50 percentage points.Source: §1.2 · p. 3
2 BACKGROUND
Prompt-based attacks manipulate natural-language context to induce unsafe behavior, often embedding harmful requests in role-playing or fictional settings. Existing defenses operate at input, model, or output stages, with different retraining and computational requirements.
- Prompt-based attacks exploit instruction-following behavior through crafted natural-language inputs rather than model parameters or internal representations.Source: §2 · p. 3
- Attackers commonly hide harmful requests inside role-playing, fictional, hypothetical, or other contextual packaging.Source: §2 · p. 3
- Input-level defenses inspect or transform prompts before target-model processing, whereas model-level defenses modify parameters or safety behavior through training.Source: §2 · pp. 3–4
- Output-level defenses examine generated responses and suppress or modify potentially unsafe content.Source: §2 · p. 4
3 THREAT MODEL
The threat model considers black-box attackers who use public interfaces and contextual manipulation to elicit harmful content. SRD-GUARD is an external, parameter-free layer that operates without access to target-model internals.
- The adversary accesses the target LLM only through its public interaction interface.Source: §3.1 · p. 4
- The adversary may use role-playing, fictional scenarios, hypothetical settings, indirect instructions, and other packaging to obscure harmful intent.Source: §3.1 · p. 4
- SRD-GUARD sits between the user and target model without modifying the target model’s parameters, architecture, or inference procedure.Source: §3.2 · p. 4
- The defense uses only the input prompt and auxiliary rewriting and scoring outputs, enabling application across target LLMs without retraining them.Source: §3.2 · p. 4
4 METHOD
SRD-GUARD exposes concealed intent through five semantic rewrites, evaluates original and rewritten prompts with multi-model risk scoring, and applies threshold- and relative-risk-based routing. The decision process intercepts high-risk requests, passes consistently low-risk requests, and warns or rewrites gray-zone cases.
4 METHOD
- SRD-GUARD generates five semantically related variants that preserve the original objective while removing unnecessary contextual packaging.Source: §4.1 · p. 4, §4.2 · p. 5
- Multiple rewritten representations reduce dependence on a single paraphrase and test whether elevated risk persists across formulations.Source: §4.2 · p. 5
- Four independent safety scorers assign continuous 0–10 risk scores to the original prompt and each rewrite, whose assessments are aggregated for consensus.Source: §4.3 · pp. 5–6
- The scoring module produces original, individual-variant, and overall rewritten-variant risks for downstream decision making.Source: §4.3 · p. 6
- Requests are intercepted when the original or any rewritten variant reaches the danger threshold S_i ≥ 6.0.Source: §4.4 · p. 6
- Requests below 3.0 for both the original and every rewrite pass unchanged, while gray-zone cases use relative risk to select a warning or rewritten response.Source: §4.4 · p. 6
5 EVALUATION
The evaluation tests SRD-GUARD’s defense effectiveness, over-refusal, component contributions, and sensitivity across harmful and benign-request benchmarks, attack methods, and target models with different intrinsic safety behavior.
- The evaluation addresses defense effectiveness, over-refusal, ablation contributions, and sensitivity to scoring-model and rewrite-variant counts.Source: §5 · p. 7
- AdvBench measures harmful requests, while transformed AdvBench prompts assess robustness against black-box jailbreak manipulation.Source: §5.1.1 · p. 7
- OR-Bench-Hard evaluates whether the defense distinguishes genuinely harmful intent from benign requests involving sensitive topics.Source: §5.1.1 · p. 7
- The study evaluates 300 AdvBench prompts and 300 OR-Bench-Hard prompts under the same defense and target-model configuration.Source: §5.1.1 · p. 7
- Llama-3-8B-Uncensored and DeepSeek-V4-Flash provide target models with substantially different intrinsic safety behaviors.Source: §5.1.2 · p. 7
- SRD-GUARD uses parameter-free semantic rewriting and four independent lightweight open-source LLM scoring models, while comparisons include DPP, IA, and SR.Source: §5.1.3 · p. 8, §5.1.4 · p. 8
5.2 ANSWER TO RQ1 (DEFENSE EFFECTIVENESS)
SRD-GUARD maintains high defense success across UNIATTACK, CIPHER, and DeepInception, including 100% DSR across all three attacks on DeepSeek-V4-Flash.
- SRD-GUARD achieves 91.00%, 85.00%, and 98.33% DSR against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored.Source: §5.2 · p. 8
- On DeepSeek-V4-Flash, SRD-GUARD achieves 100% DSR across UNIATTACK, CIPHER, and DeepInception.Source: §5.2 · p. 8
- SRD-GUARD outperforms DPP and SR on Llama-3-8B-Uncensored while remaining competitive with IA across the three attacks.Source: §5.2 · p. 8
- SRD-GUARD improves DSR over the strongest baseline by 16.67 percentage points on CIPHER and 10.77 percentage points on DeepInception.Source: §5.2 · p. 8
- The defense combines semantic rewriting, joint scoring, and absolute-plus-relative risk routing to detect concealed harmful requests.Source: §5.2 · p. 9
- SRD-GUARD increases ORR by 2.33% on Llama-3-8B-Uncensored and 2.00% on DeepSeek-V4-Flash while improving DSR across the attacks.Source: §5.2 · p. 9
5.3 ANSWER TO RQ2 (OVER-REFUSAL)
SRD-GUARD achieves a favorable defense–utility balance, combining low over-refusal with strong DSR; ablations show that rewriting, joint scoring, and risk-adaptive decisions contribute complementarily.
- SRD-GUARD records 8.00% ORR on Llama-3-8B-Uncensored and 12.00% on DeepSeek-V4-Flash, below IA’s 45.67% and 38.33%.Source: §5.3 · p. 9
- On Llama-3-8B-Uncensored, SRD-GUARD pairs DSRs of 91.00%, 85.00%, and 98.33% with 8.00% ORR across the three attacks.Source: §5.3 · p. 9
- SRD-GUARD improves average DSR by 30.30% on Llama-3-8B-Uncensored and 15.67% on DeepSeek-V4-Flash while reducing ORR relative to baselines.Source: §5.3 · p. 10
Ablation study
- The complete system achieves the highest average DSR and lowest ORR across both target models, while removing any component degrades one or both objectives.Source: §5.4 · p. 10
- Removing rewriting drops average DSR from 91.44% to 50.66% on Llama-3-8B-Uncensored and from 100% to 64.52% on DeepSeek-V4-Flash.Source: §5.4 · p. 10
- Replacing joint scoring with one Llama3-8B scorer raises ORR to 37% and 50% on the two target models despite an apparent DSR increase in one setting.Source: §5.4 · p. 10
- Removing the Decision Module lowers average DSR to 79.33% and 79.50% while raising ORR to 24.50% and 32.50%.Source: §5.4 · p. 11
- The three components have complementary roles: rewriting exposes hidden intent, joint scoring stabilizes risk estimation, and decision routing controls over-refusal.Source: §5.4 · p. 11
5.5 ANSWER TO RQ4 (SENSITIVITY)
Sensitivity analysis identifies five rewritten variants and four scoring models as the strongest configuration, balancing defense success against over-refusal.
- 85% DSR and 8% ORR make five rewritten variants the strongest tested configuration.Seven variants reduce DSR to 82% and increase ORR to 12%.Source: §5.5 · p. 11
- 85% DSR and 8% ORR make four scorers the strongest tested scoring configuration.The passage states that using fewer scorers produces weaker performance.Source: §5.5 · p. 11
- Table 2 reports sensitivity analysis on Llama-3-8B-Uncensored under CIPHER.Source: Table 2 · p. 11, §5.5 · p. 11
6 DISCUSSION
The discussion identifies deployment cost and unresolved safety-boundary ambiguity as important constraints on SRD-GUARD’s practical use.
Deployment Overhead
- Four independent LLM evaluators create substantial memory and computation overhead during deployment.They score both the original prompt and rewritten variants using continuous risk scores.Source: §6 · p. 11
Ambiguity of Safety Boundaries
- OR-Bench’s sensitive, contextually legitimate prompts make a sharp harmfulness boundary difficult to establish.Source: §6 · pp. 11–12
- SRD-GUARD cannot simultaneously achieve zero over-refusal and perfect defense success on borderline requests.Increasing sensitivity to concealed harmful intent may affect some benign requests, while relaxing the defense may miss some harmful requests.Source: §6 · p. 12
7 RELATED WORK
Related defenses use perturbation, intention analysis, or multi-model judging, but SRD-GUARD combines multiple semantic representations with continuous, relative-risk-aware scoring.
- Input-level defenses such as SmoothLLM primarily manipulate suspicious prompts and may increase over-refusal through conservative treatment.Source: §7 · p. 12
- Intention Analysis infers a single essential intention rather than comparing the original prompt with multiple semantic representations.Source: §7 · p. 12
- Existing multi-model defenses generally assess one prompt representation with absolute judgments, whereas SRD-GUARD jointly scores originals and rewrites.SRD-GUARD adds continuous risk scoring and explicit relative-risk comparison.Source: §7 · p. 12
8 CONCLUSION
SRD-GUARD exposes concealed intent through semantic rewriting, aggregates independent safety judgments, and routes requests using absolute and relative risk. Across target models, it achieves strong defense effectiveness while preserving benign-request utility more selectively than uniformly restrictive defenses.
- SRD-GUARD is a parameter-free, black-box framework that uses semantic rewriting, joint scoring, and risk-adaptive routing.The decision module combines absolute risk with relative risk changes between original and rewritten prompts.Source: §8 · p. 12
- 91.44% average DSR on Llama-3-8B-Uncensored and 100% on DeepSeek-V4-Flash accompany ORRs of 8.00% and 12.00%, respectively.Source: §8 · p. 12
- The framework targets a more favorable balance between defense effectiveness and benign-request preservation without modifying the target model.Source: §8 · p. 13