Source-linked AI summary
BRANCH: Bypassing Multi-Scanner AI Guardrails
William Hackett, Peter Garraghan
TL;DR
Established evasion techniques struggle to bypass guardrail systems with multiple scanners, a gap that matters because scanners cover different malicious-content categories. BRANCH uses branching search to target scanners individually and reports 100% bypass success across six target systems with fewer queries and less time; generated bypasses also transfer to unseen guardrails.
Problem
Established evasion techniques struggle to bypass all scanners in a guardrail system simultaneously or succeed only at substantial computational cost.
Source: §1 · p. 1Method
BRANCH uses branching tree search to isolate perturbations against individual scanners and select branches that gradually bypass the system’s scanners.
Source: §1 · pp. 1–2, §4.3 · p. 6Results
100% bypass success across 6 target guardrail systems and 120 scenarios, using 72% fewer queries and 4.5x less wall-clock time per bypass.
Source: §9 · p. 13Takeaways & Limitations
BRANCH-generated bypasses transferred to 29 unseen production guardrails without further perturbation, including 8 commercial black-box systems.
Source: §1 · p. 2The study does not thoroughly evaluate bypassed prompts’ efficacy against production LLMs.
Source: §7 · p. 13
Abstract
from arXiv · showhide
AI systems increasingly rely on Large Language Models (LLMs) as core reasoning engines, making them targets for prompt injection and jailbreaks. Guardrails monitor and validate model inputs and outputs, yet their isolated, task-focused detection leaves gaps in their classification making them susceptible to bypasses. In response, guardrail systems formed by multiple scanners have emerged that collaboratively detect different types of malicious instructions, whereby shared latent representations across classification boundaries render established bypassing techniques ineffective. We propose BRANCH, a bypassing methodology designed for multi-scanner guardrail systems. Our method leverages a branching tree search approach that dynamically applies adversarial perturbation against individual scanners, with subsequent perturbation optimization and technique selection based on overall improvement across all guardrail system scanners, effectively decoupling bypass evaluation from attack signal optimization. Our findings demonstrate that BRANCH achieves 100% attack success rate across 6 guardrail systems in 120 scenarios with 72% fewer queries and 4.5x reduced wallclock time compared to established techniques, while preserving semantic meaning within the bypass. We also show how bypasses generated by BRANCH transfer to 29 unseen guardrails, including 8 commercial black-box guardrails, improving attack success in some cases up to 100% with no additional optimization.
1 Introduction
Multi-scanner guardrails require attacks to evade several scanners, but established techniques struggle with that joint objective. BRANCH searches across scanner-specific perturbations and achieves high bypass success with fewer queries and less time, with transfer to unseen guardrails.
1 Introduction
- Established evasion techniques struggle to bypass multiple scanners simultaneously, achieving lower success at substantial computational cost.The introduction identifies this as an unaddressed research gap for attacks against multi-scanner systems.Source: §1 · pp. 1–2
- BRANCH chains perturbation techniques while isolating bypassing to individual scanners and selecting branches that gradually evade all scanners together.The approach is described as applicable to any malicious instruction and perturbation-agnostic.Source: §1 · pp. 1–2
- 100% bypass success across 6 target guardrail systems, using 72% fewer queries and 4.5x less wall-clock time than the state-of-the-art.The comparison also reports semantic content comparable to established techniques.Source: §1 · p. 2
- BRANCH bypasses exposed an average 32.9-percentage-point increase in attack success rate across 29 unseen production guardrails without further perturbation.The 29 include 8 commercially offered black-box guardrail systems.Source: §1 · p. 2
2 Background
The background distinguishes single-category scanners from guardrail systems that combine scanners, then reviews adversarial methods for modifying inputs to evade detection.
2.1 Guardrails
- A scanner detects a single content category, whereas a guardrail system combines scanners to make a classification decision across threat categories.Scanners may use classifiers, regular expressions, pattern matching, or LLM-as-a-judge methods.Source: §2.1 · p. 2
2.2 Bypassing Techniques
- Adversarial evasion adapts input perturbations to bypass guardrails, including genetic algorithms, token replacement, character changes, and word substitutions.Word importance ranking prioritizes tokens whose removal most changes the guardrail’s confidence.Source: §2.2 · pp. 2–3
- Figure 1 scopes static scanner activations across 8 LLM Guard scanners on 3,191 prompt-injection and jailbreak samples.The samples come from PIGuard.Source: Figure 1 · p. 3
3 The Challenges of Established Bypassing Techniques Against Guardrail Systems
Established single-objective techniques face overlapping scanner boundaries and interference when they try to bypass a multi-scanner system. Perturbations can raise another scanner’s confidence, while scalar aggregation obscures scanner-specific feedback.
3 The Challenges of Established Bypassing Techniques Against Guardrail Systems
- Shared scanner features can create overlapping decision boundaries, while collapsing per-scanner confidences into one scalar can cause interference and obscure scanner-specific feedback.A perturbation that lowers one scanner’s confidence can raise another’s, and a scanner not initially blocking may become the dominant blocker.Source: §3 · p. 3, §3.1 · pp. 3–4, §3.2 · p. 4
3.2 Scanner Interference
- PWWS’s dominant blocking scanner switched 14 times over 1,637 queries, while TextBugger’s switched 7 times over 1,033 queries.Figure 2 tracks scanner confidences and the currently highest blocking scanner during perturbation.Source: Figure 2 · p. 4, §3.2 · p. 5
3.1 Overlapping Decision Boundaries
- 20.15% of 3,191 static samples activated two or more scanners, and 7 of 8 LLM Guard scanners activated at least once.The samples were labeled prompt injection or jailbreak; the authors note these measurements precede perturbation, which may activate another scanner.Source: §3.1 · p. 4
4 BRANCH
BRANCH turns multi-scanner bypassing into a sequence of single-scanner perturbations, selecting and validating candidates against the guardrail system as a whole. The threat model assumes access to per-scanner decisions and confidence scores, but not model internals.
4 BRANCH
- BRANCH targets one blocking scanner at a time, scores candidates by total blocked confidence, and chains perturbations to lower confidence across the system as a whole.The search expands the pending node with the lowest total blocked confidence.Source: §4 · p. 5
4.1 Threat Model
- The adversary seeks prompts that evade every scanner while preserving the seed prompt’s malicious intent, with black-box access to per-scanner decisions and confidence scores.The threat model assumes no access to model weights, gradients, internal architectures, or training data, and imposes no budget or rate limit.Source: §4.1 · p. 5
4.2 Preliminaries
- A prompt is blocked when any scanner blocks it; BRANCH tracks the summed confidence of blocking scanners and targets the blocking scanner with the highest confidence.The search tree stores each prompt, its target scanner, total blocked confidence, and depth; candidate prompts are modified variants generated against the target scanner.Source: §4.2 · p. 6
4.3 Methodology
- BRANCH selects the pending node with the lowest total blocked confidence, then perturbs a target scanner, validates candidates against the full system, and grows the tree with blocked candidates.The process can chain perturbation techniques across nodes; it ends when a bypass is found or the search is exhausted.Source: §4.3 · pp. 6–7, Figure 3 · p. 7
5 Experimental Setup
The evaluation compares BRANCH with individual and exhaustive applications of 11 bypass techniques across six multi-scanner guardrail systems. It uses 120 seed-prompt scenarios and measures success, efficiency, similarity, and adversarial utility.
Evaluation approaches
- BRANCH chains techniques across a search tree, while individual and exhaustive baselines apply techniques separately; the baselines have unbounded query budgets, unlike BRANCH’s per-node and tree limits.BRANCH uses up to 5,000 queries per node, with Nmax = 120 and Dmax = 120.Source: §5 · p. 7
Targets
- Six guardrail systems are evaluated, using default confidence thresholds or 0.50 when no default is provided; one is a custom ensemble of three open-source models.The custom ensemble covers prompt injection and jailbreak, toxicity, and gibberish detection.Source: §5 · p. 8, Table 1 · p. 8
Bypass Candidates
- The evaluation uses 120 scenarios: 20 malicious prompts per system, selected from PIGuard for being blocked by the most scanners across the targets.Experiments were run serially on a Linux Ubuntu 20.04 server with 128GB RAM, an Intel Xeon Gold 6336Y CPU, and two A100 GPUs.Source: §5 · p. 8
Evaluation Metrics
- Attack success rate, queries, and wall-clock time measure bypass performance, while surface similarity, semantic preservation, and judged output similarity assess bypass fidelity and utility.Semantic preservation uses cosine similarity between sentence embeddings; an LLM judge compares outputs from an ablated Qwen 3 8B model on original and bypass prompts.Source: §5 · p. 8
6 Results
BRANCH bypasses all six target guardrail systems more successfully and efficiently than the comparison methods, while retaining semantic content. Its bypasses also transfer to unseen guardrails, including commercial black-box offerings.
6.1 Bypass Effectiveness
- 100% of 120 scenarios were bypassed by BRANCH, versus 80.8% by Exhaustive; BRANCH succeeded across all six target guardrail systems.Figure 4 reports overall attack success rates across the six systems.Source: §6.1 · p. 8, Figure 4 · p. 9
- 219,138 queries per bypass for BRANCH versus 786,269 for Exhaustive represents a 72.13% reduction; wall-clock time was 2.9h versus 13.1h, a 4.51x speed-up.Several individual techniques used fewer queries than BRANCH but had substantially lower ASR; query cost was highest against Ensemble Guardrail.Source: Figure 5 · p. 9, §6.1 · pp. 9–10
- Chaining let lower-performing techniques contribute on BRANCH’s optimal paths: BAE use rose from 13 to 115 and Pruthi from 4 to 41, while Kuleshov fell from 70 to 41.The authors attribute this to chained techniques sharing optimizations and combining advantages of different perturbation approaches.Source: §6.1 · p. 10, Figure 6 · p. 10
6.2 Generated Bypass Similarity
- BRANCH’s average semantic preservation was comparable to Exhaustive’s, while average surface similarity was 0.60 versus 0.71.For LLM Guard, semantic preservation was 0.86 for BRANCH versus 0.87 for Exhaustive; Figure 7 plots semantic preservation against ASR.Source: §6.2 · p. 10, Figure 7 · p. 11
- As tree depth increased, surface similarity fell more sharply than semantic preservation, which typically plateaued after the first few layers.On Guardrails AI, surface similarity fell from 1.0 to 0.23 by depth 21; on Samsung Guard, it reached 0.14 by depth 13.Source: §6.2 · p. 11
- 85% of BRANCH bypasses scored at least 3 in adversarial utility versus 90% for Exhaustive, while 54% scored 5 versus 51%.The utility scores compared outputs from an ablated Qwen 3 8B model for generated bypasses and original seed prompts.Source: §6.2 · p. 11, Figure 8 · p. 11
6.3 Transferability
- 29 of 32 unseen guardrails were bypassed, with per-scanner ASR ranging from 5% to 100%; transfer varied by target pairing.Five scanners had 100% ASR, while the authors describe shared weaknesses among particular targets rather than a generalized weakness across scanners.Source: §6.3 · p. 12, Figure 9 · p. 12
- BRANCH-generated prompts increased ASR by an average of 32.9 percentage points across unseen guardrails, including eight evaluated commercial black-box offerings.The reported increases for those commercial offerings ranged from 5% for Lakera Guard to 90% for Azure Content Safety.Source: §6.3 · p. 12
7 Limitations
The evaluation is bounded by its selected prompts, while the threat model assumes per-scanner outputs and the study does not thoroughly test bypassed prompts against production LLMs.
7 Limitations
- The evaluation uses 20 prompts per guardrail system, selected because they were blocked by the most scanners among the targets.The authors suggest extending the study to prompts from other datasets.Source: §7 · pp. 12–13
- BRANCH uses a fixed query budget of 5,000 per node, which the authors identify as an opportunity for further optimization.They suggest adapting the budget to parent nodes and perturbation techniques to potentially reduce queries and search depth.Source: §7 · p. 13
- BRANCH’s generation assumes access to per-scanner confidence scores and blocking decisions, which may be hidden in strict black-box deployments.The authors note that generated bypasses can nevertheless transfer offline to unseen targets without optimization.Source: §7 · p. 13
- The study does not thoroughly test whether bypassed prompts are effective against production LLMs.The authors identify this as further work, including assessing effects on prompt attack success rates.Source: §7 · p. 13
8 Related Work
Prior evasion research has largely evaluated attacks on individual scanners, leaving multi-scanner guardrail systems—where every scanner must be evaded—as a distinct setting.
8 Related Work
- Earlier adversarial evasion work targeted text-classification models and was later applied to guardrail classifiers, with evaluations focused on isolated single-scanner guardrails.Examples include TextFooler, BERT-Attack, and PWWS.Source: §8 · p. 13
- Existing studies do not evaluate systems containing multiple scanners that all must be bypassed for an attack to succeed.More recent approaches exploit input-window hiding or representation asymmetry between guardrail classifiers and the target model.Source: §8 · p. 13
9 Conclusion
The conclusion presents BRANCH as a method for bypassing multi-scanner guardrail systems, with high success across evaluated targets and transfer to unseen systems.
9 Conclusion
- BRANCH achieved 100% bypass success across 6 target guardrail systems and 120 bypass scenarios, consuming 72% fewer queries and 4.5x less wall-clock time per bypass.Source: §9 · p. 13
- BRANCH-generated bypasses transfer reasonably well to many unseen guardrail systems, including commercial offerings.Source: §9 · p. 13
- The authors frame the findings as useful both for developing evasion techniques for more sophisticated defenses and for helping practitioners evaluate guardrail effectiveness.Source: §9 · p. 13
A.1 The Challenges of Single-Objective Techniques Against Guardrail Systems
Figure A.1 describes how many scanners simultaneously flag candidate prompts across target guardrail systems.
A.1 The Challenges of Single-Objective Techniques Against Guardrail Systems
- Figure A.1 reports the distribution of the number of scanners that simultaneously flag each of 3,191 candidate prompts, broken down by target guardrail system.Source: Figure A.1 · p. 17
A.2 Experimental Setup
The appendix outlines the perturbation techniques and target-detector categories used in the experiments, then presents figures and a table covering query efficiency, bypass similarity, and transferability.
A.2 Experimental Setup
- Table A.1 lists the 11 perturbation techniques used in Exhaustive and BRANCH, while Table A.2 describes the detector categories of the experimental targets.Source: Table A.1 · p. 18, Table A.2 · p. 19
A.3 Query Efficiency and Technique Composition
- Figure A.2 compares average wall-clock time per bypass job across BRANCH, Exhaustive, and 11 individual techniques over six target guardrail systems.Source: Figure A.2 · p. 19
- Figure A.3 shows per-prompt bypass coverage for each technique across different guardrail systems.Source: Figure A.3 · p. 20
A.4 Bypass Similarity
- Table A.3 reports semantic preservation and attack success rate by technique and target; Figure A.4 presents average surface similarity and semantic preservation against the original seed prompt.The table defines n as successful bypasses, Pres. as mean semantic preservation ± 1 standard deviation, and ASR over 20 seed prompts.Source: Table A.3 · p. 20, Figure A.4 · p. 21
- Figure A.5 tracks BRANCH surface similarity by depth across bypass scenarios and targets, while Figure A.6 tracks semantic preservation across the same scope.Source: Figure A.5 · p. 21, Figure A.6 · p. 22
A.5 Transferability
- Figure A.7 is a transferability heatmap showing the percentage of bypasses from an original target that transfer to an unseen guardrail.Source: Figure A.7 · p. 23