SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration

1 Introduction

• Strong native alignment does not imply jailbreak robustness, and defended attack success rate alone can miss marginal security gains and their utility and efficiency costs.Automated and optimization-based attacks continue to bypass aligned models using adversarial prompts.Source: §1 · p. 2
• A safe final response does not guarantee safe planning, memory, tool interaction, or execution, because agentic attacks can enter through indirect content, persistent memory, tools, and inter-agent messages.The paper distinguishes an attack’s entry vector from its objective and counts prompt injection as a jailbreak only when it bypasses an explicit safety policy.Source: §1 · p. 2
• The SoK develops a multidimensional taxonomy of agentic jailbreak attacks and defenses spanning access, injection channels, target components, persistence, interaction, optimization, security properties, trust boundaries, and defense stages.Its scope includes adversarial surfaces across planning, tools, memory, and multi-agent interaction.Source: §1 · p. 3
• The evaluation framework separates harmful-prompt safety, adversarial jailbreak robustness, and agent-level security while accounting for over-refusal, response quality, task success, latency, and computational cost.The framework addresses the need to evaluate defenses beyond model-level attack outcomes.Source: §1 · pp. 2–3
• The controlled evaluation of inference-time defenses with Qwen3.5-4B and Llama-3.1-8B-Instruct finds substantial model-, attack-, and component-dependent differences in robustness.The paper frames its experiments as representative model-layer defenses under a common setting, not as evidence of complete agent-level protection.Source: §1 · p. 3

2.1 Literature Search

• The literature search covered work from 2022–2026 and screened 271 attack-related and 88 defense-related candidates into a taxonomy of 116 attacks and 63 defenses.The search used multiple scholarly sources, citation tracing, manual screening, and systematic coding.Source: §2.1 · pp. 3–4

2.2 Existing Jailbreak and Agentic AI Studies

• Earlier jailbreak surveys center on standalone model interactions, while agent-security surveys cover broader system threats, trust boundaries, and component interactions [169] [150] [132] [23] [30] [24].The closest prior SoKs either focus on model-level guardrails or treat jailbreak as one threat within agentic security [169] [150].Source: §2.2 · p. 4
• The empirical design evaluates two final models and nine attacks under shared implementation and judging procedures, measuring final and intermediate safety alongside utility and efficiency.Outcomes include unsafe planning, reasoning, memory, and tool interaction, over-refusal, response quality, task utility, latency, and computational cost.Source: §2.2 · pp. 4–5
• The SoK connects a jailbreak-specific taxonomy with controlled cross-layer evaluation, linking component-level failures to final safety and practical trade-offs.Its unit of analysis is the agent trajectory rather than only the prompt–response pair.Source: §2.2 · pp. 4–5
• Table 1 compares taxonomy scope, unit of analysis, measured outcomes, and empirical design descriptively, using coverage categories rather than a ranking.Comprehensive, partial, and not covered indicate whether an aspect has a dedicated taxonomy, framework, or empirical study, is discussed but not central, or lacks substantive treatment.Source: §2.2 · p. 5, Table 1 · p. 5

3.1 Agentic AI Systems

• Agentic systems extend conversational LLMs with task decomposition, persistent memory, tool use, and multi-agent collaboration for complex tasks.They can adapt to changing environments and pursue multi-step objectives with limited human intervention.Source: §3.1 · p. 5

3.2 Agent Execution Pipeline

• Agents iteratively interpret objectives, decompose subtasks, retrieve information, invoke tools, and use observations to guide later decisions; components and transitions can be manipulated.Multi-agent systems may also exchange information and coordinate actions.Source: §3.2 · p. 5, Figure 1 · p. 6

3.3 Agentic Threat Model

• The threat model distinguishes black-box control of inputs and external content from stronger white-box access, and separates final-response jailbreaks from unsafe intermediate behavior.Intermediate components may become unsafe even when the final response remains safe.Source: §3.3 · p. 6

4.1 Taxonomy of Agentic Jailbreak Attacks

• Attacks are grouped by access: black-box methods by targeted interaction surface and white-box methods by optimization mechanism.The attack taxonomy is shown in Fig. 2.Source: §4.1 · p. 6, Figure 2 · p. 7
• User, reasoning, and tool attacks range from prompt refinement and trajectory manipulation to executable tool chains, with evaluations often limited in scope.Reported constraints include query or computational cost, restricted model or language coverage, and limited evaluation across agents and tools.Source: §4.1 · pp. 7–8
• Memory and inter-agent attacks can persist or propagate beyond the initial interaction, but memory evidence is limited and attacks rely on system-specific assumptions.MINJA depends on later retrieval of injected memory; other attacks assume conditions such as repeated debate, randomized interactions, or executable tools.Source: §4.1 · p. 8
• White-box attacks use gradient, search, or representation-aware optimization, but generally require privileged access and may transfer poorly across architectures or aligned models.These approaches can also incur costly optimization and depend on internal features or signals.Source: §4.1 · pp. 8–9

4.2 Agentic Jailbreak Defense Taxonomy

• Defense categories span context and state, planning and orchestration, tools and execution, and inter-agent communication, alongside model-and-reasoning defenses.Fig. 3 organizes defenses by the architectural layer primarily protected; system-level mechanisms are included when they prevent, detect, or contain policy bypass.Source: Figure 3 · p. 9, §4.2 · pp. 10–11
• Training, input transformation, and internal-state controls intervene at different model stages but can require model access, extra inference, or repeated checking.Their limitations include reduced transfer, over-refusal, overhead, and difficulty with adaptive or later-introduced attacks.Source: §4.2 · pp. 10–11
• Output classifiers and revisers may over-refuse or miss concealed intent, and cannot reliably observe or reverse unsafe actions that occurred earlier in an agent trajectory.This includes unsafe plans, memory writes, tool calls, or external side effects.Source: §4.2 · p. 11
• Tool defenses mediate consequential calls through least-privilege policies or causal attribution, while inter-agent defenses mainly verify or monitor rather than control communication comprehensively.Their constraints include runtime integration, false positives in some workflows, and limits on remediation or enforcement.Source: §4.2 · pp. 11–12

5.1 Experimental Setup

• A fixed LangGraph agent framework benchmarks 16 defenses and Vanilla across XSTest, AdvBench, and AgentHarm, measuring security, utility, efficiency, and component-level unsafe behavior.The setup evaluates final-response models Qwen3.5-4B and Llama-3.1-8B-Instruct, with Qwen3.5-4B as the router/tool model unless otherwise specified.Source: §5.1 · p. 13

5.2 Practical Trade-off Analysis

• For Qwen3.5-4B, Self-Reminder ranks first overall at 86.8; for Llama-3.1-8B-Instruct, Self-Eval leads at 97.1, showing the preferred defense varies by model.Self-Reminder reduces Qwen ASR from 3.67% to 1.01% and achieves 81.62% aggregate utility; SafeProbing has 1.00% ASR but takes 140.68 s versus Vanilla’s 72.44 s. Prompt Guard and Robust Aligned reach 0% ASR but reject all benign XSTest prompts, yielding 100% over-refusal and 0% utility.Source: §5.2 · pp. 14–15, Figure 5 · p. 15

5.4 Impact of Agentic AI Components

• Component-level attacks can severely compromise planning and memory even when final-response ASR appears modest.MINJA produces more than 91% unsafe memory and 100% poison retrieval, while TRACE raises unsafe planning to 91.49% despite final ASR of 11.54%. Self-Eval reaches 0% final ASR under all three attacks, yet unsafe memory reaches 95.65% under MINJA and unsafe planning 93.62% under TRACE; unsafe tool-interaction rates also remain high, and Self-Eval has the highest latency.Source: §5.4 · pp. 17–20

5.3 Jailbreak Robustness in Agentic AI

• On AdvBench, Vanilla averages 34.00% ASR and Prompt Guard 0%, but defenses remain attack-dependent, with ICD reaching 66.00% on AutoDAN and 46.00% on CodeChameleon.The evaluation uses nine representative jailbreak attacks, and the results show that performance on one attack does not establish robustness across attack mechanisms.Source: §5.3 · p. 15, Table 8 · p. 21

5.5 Overview of Jailbreak Attacks and Defenses in Agentic AI

• The section organizes attacks by access, surface, interaction, evaluation, mechanism, availability, and limitations, and defenses by mechanisms, protected components, settings, security, utility, and limitations.Source: §5.5 · pp. 21–22
• Agent-agent attacks use role specialization or inter-agent infection, while some require multiple attacker agents, iterative testing, or target role definitions [141], [41].The surveyed methods include role-specialization rewriting and infectious jailbreaks; executable tool access is also identified as an assumption for command propagation.Source: §5.5 · pp. 24–25
• GCG [213] uses greedy coordinate gradient optimization and requires gradient access to source models; Faster-GCG [76] requires white-box access to model parameters, and suffix optimization is costly and iterative.Source: §5.5 · p. 25
• User-agent prompt attacks include genetic optimization [106] and token-aware mutation guided by surrogate refusal signals [18], but are limited to single turns or require a local white-box surrogate.The overview describes LASH as adaptive composition of jailbreak prompts and TriageFuzz as token-aware prompt mutation.Source: §5.5 · p. 25
• Emoji insertion [158] exploits token-segmentation bias in judge models, while its effectiveness varies across judges; PiF [81] uses token replacement to obscure malicious intent.The overview notes that PiF lacks a comprehensive theory of jailbreak importance.Source: §5.5 · p. 26
• SEQAR [177] auto-generates sequential role-playing characters for jailbreak prompts, while ArtPrompt [58] uses ASCII-art masking to hide sensitive words.Source: §5.5 · p. 27
• DeGCG [85] uses two-stage gradient search for adversarial suffix transfer, while COLD-Attack [42] uses Langevin sampling over continuous logits to generate fluent, controllable jailbreaks.COLD-Attack is computationally costly and less effective when safety system prompts are present.Source: §5.5 · pp. 27, 31
• h4rm3l [34] synthesizes target-specific attacks with bandit-based program search, while RLbreaker [19] uses DRL to select prompt mutators but may miss valid responses that differ from references.Source: §5.5 · p. 27
• ASETF reports lower ASR for universal suffixes, trading off semantic coherence.It optimizes continuous suffix embeddings and translates them into fluent strings.Source: §5.5 · p. 27
• CodeAttack [122] reformulates harmful requests as code-completion tasks, but its evaluation is code-domain-only and generalization to other non-natural-language domains remains unexplored.Source: §5.5 · p. 27
• SecDecoding depends on auxiliary-model quality, SSD [151] is limited to small compatible expert models, and AED requires self-evaluation at each decoding step.Source: §5.5 · p. 33
• PAT [104] loses effectiveness against adaptive attacks with defense knowledge, THRD [96] incurs substantial multi-turn inference overhead, and RLM-JB [134] remains vulnerable to adaptive detector-targeting attacks.Source: §5.5 · p. 35
• R2D [209] is evaluated mainly on single-turn text jailbreaks, leaving multi-turn and multimodal safety unexplored; SafeDecoding [172] may miss later harmful transitions and lacks multimodal evaluation.SafeDecoding also requires a fine-tuned expert model.Source: §5.5 · p. 35
• Fixed steering parameters may not adapt across contexts and can be exploited by adaptive white-box attackers.Source: §5.5 · p. 35
• Backtranslation [152] relies on a safety-aligned base model and may fail to recover hidden harmful intent, while hidden-state jailbreak classification requires model-specific retraining and depends on base-model alignment.Source: §5.5 · p. 35
• Execution-control/data-flow separation assumes a trusted query and maintained security policies, yet side channels and compositions of individually permitted control flows remain possible.Source: §5.5 · p. 35

• Strong native safety does not establish jailbreak robustness, so evaluations should distinguish baseline harmful-prompt safety from adversarial robustness.Source: §6 · p. 37
• Defenses should intervene proportionally on suspicious requests and high-consequence actions while preserving normal functionality, rather than broadly refusing benign requests.Source: §6 · p. 37
• Robustness testing should span models, attacks, and interaction settings, including adaptive adversaries and attacks that accumulate influence across turns or components.Source: §6 · pp. 37–38
• Agentic defenses should protect component states and transitions so unsafe intermediate behavior does not propagate into persistent memory, privileged tool calls, or external actions.Source: §6 · p. 37
• Persistent memory calls for source authentication, provenance tracking, controlled write permissions, retrieval validation, and safe revocation or quarantine.Source: §6 · p. 37
• High-risk tool operations should require explicit authorization that considers parameters, context, expected consequences, and cumulative effects across actions.Source: §6 · p. 37
• Defenses should combine lightweight monitoring with specialized safeguards at high-risk boundaries; protecting one component alone does not secure the complete agent.Source: §6 · pp. 37–38
• Future benchmarks should measure intermediate and end-to-end outcomes, including unsafe planning, malicious state persistence, and unauthorized tool execution, alongside utility, latency, and computational cost.Source: §6 · pp. 37–38

• Jailbreak security in agentic AI spans reasoning, memory, tools, and multi-agent interaction, rather than only model outputs.Source: §7 · p. 38
• A safe final response can coexist with compromised planning, memory, or tool interactions, making response-level safety an incomplete measure.Source: §7 · p. 38
• Modern models provide a strong native safety baseline, while added defenses may yield marginal, model- and attack-dependent gains with over-refusal, utility loss, and computational overhead.Source: §7 · p. 38

• The study evaluates publicly documented attacks and defenses in an isolated or simulated setting, without human subjects, private user data, or unauthorized interaction with deployed systems.Source: Ethical Considerations · p. 38
• To mitigate possible attack-reproduction risks, the authors exclude operationally sensitive deployment details, prevent real-world tool effects, and emphasize defensive findings and mitigations.Source: Ethical Considerations · p. 38
• Future studies should protect researchers from distressing generated content through warnings, opt-out procedures, workload rotation, limited exposure, and support resources.Source: Ethical Considerations · p. 39

• Md. Jueal Mia is pursuing a computer science Ph.D. at Florida International University and researches trustworthy AI, AI security, foundation models, agentic AI, federated learning, and privacy-preserving machine learning.Source: Author Biographies · p. 50
• Yanzhao Wu is an assistant professor at Florida International University whose interests include machine-learning systems, large language models, generative AI, and privacy-preserving machine learning.Source: Author Biographies · p. 50
• Selcuk Uluagac is an eminent scholar chaired professor at Florida International University and directs CIERTA and the Cyber-Physical Systems Security Lab.Source: Author Biographies · p. 50
• M. Hadi Amini is an associate professor at Florida International University, directs the solid lab, and serves as associate director of the USDOT National Center for Transportation Cybersecurity and Resiliency.Source: Author Biographies · p. 51